feat(python): add Python IDE helper generation and update object conversion methods

- Add Command class to handle --gen-python-helper and --convert-python-to-php operations
- Integrate PythonToolsCommand execution into main compiler flow
- Remove toPlainValue from CompilerBase keyword method map
- Update tests to use new toValue() and toArray() methods instead of toPlainValue()
- Rename plain-value tests to object-conversion-methods with updated expectations
- Replace toPlainValue documentation with toValue() and toArray() explanations
- Add HelperRenderer to generate PHP declarations for Python modules
- Implement PhpyModuleScanner to inspect Python modules via reflection
- Add PyObjectHelperRenderer to generate base PyObject IDE helper
- Create example pi.php demonstrating new Python integration features
- Update gitignore to exclude ide-helper directory
pull/48/head
韩天峰 2 weeks ago
parent f61a542255
commit 6c97b4f301
  1. 1
      .gitignore
  2. 10
      README.md
  3. 44
      docs/python/design.md
  4. 4
      docs/python/implementation-plan.md
  5. 86
      docs/python/tools.md
  6. 13
      examples/python/pi.php
  7. 15
      phpunit/code/python/object-conversion-methods.php
  8. 10
      phpunit/code/python/plain-value-arguments.php
  9. 15
      phpunit/code/python/plain-value.php
  10. 3
      phpunit/src/CompilerBaseApiTest.php
  11. 23
      phpunit/src/Python/PythonModuleTest.php
  12. 156
      phpunit/src/PythonTools/PythonIdeHelperTest.php
  13. 76
      phpunit/src/PythonTools/PythonToTypePhpConverterTest.php
  14. 36
      phpunit/src/PythonTools/PythonToolsCommandTest.php
  15. 1
      src/CompilerBase.php
  16. 13
      src/Parser/MethodCallTrait.php
  17. 1
      src/Parser/UniversalMethodCall.php
  18. 145
      src/PythonTools/Command.php
  19. 91
      src/PythonTools/Converter/PythonAstLoader.php
  20. 648
      src/PythonTools/Converter/PythonToTypePhpConverter.php
  21. 176
      src/PythonTools/IdeHelper/HelperRenderer.php
  22. 120
      src/PythonTools/IdeHelper/PhpyModuleScanner.php
  23. 65
      src/PythonTools/IdeHelper/PyObjectHelperRenderer.php
  24. 2
      src/Translator.php
  25. 9
      src/compiler.php
  26. 41
      tests/compiler/python/object-conversion-methods.phpt
  27. 53
      tests/compiler/python/plain-value.phpt

1
.gitignore vendored

@ -31,3 +31,4 @@ tests/**/*.php
tests/**/*.sh tests/**/*.sh
/*.browser/ /*.browser/
/tests/wasm/harness/node_modules/ /tests/wasm/harness/node_modules/
/ide-helper/

@ -26,6 +26,16 @@ bin/tpc.php project.yml
Linux 环境缺少 `libphp.so` 时,`tpc.php` 可以交互式下载 PHP 源码并自动构建,详见 [自动构建 libphp.so](docs/LIBPHP_INSTALLER.md)。 Linux 环境缺少 `libphp.so` 时,`tpc.php` 可以交互式下载 PHP 源码并自动构建,详见 [自动构建 libphp.so](docs/LIBPHP_INSTALLER.md)。
Python namespace IDE helper 与 Python 源码转换也使用同一入口:
```shell
./tpc --gen-python-helper math
./tpc --gen-python-helper numpy --output-dir .ide-helper
./tpc --convert-python-to-php script.py > script.php
```
详细说明见 [Python 工具子模块](docs/python/tools.md)。
```shell ```shell
# Ubuntu/Debian # Ubuntu/Debian
sudo apt install libgmp-dev libmpfr-dev libmpdec-dev sudo apt install libgmp-dev libmpfr-dev libmpdec-dev

@ -398,7 +398,7 @@ python\setattr($os, 'name', $value);
```php ```php
python\print('hello'); // 等价于 PyCore::print('hello') python\print('hello'); // 等价于 PyCore::print('hello')
$length = python\len($value)->toPlainValue()->toInt(); $length = python\len($value)->toValue()->toInt();
$range = python\range(0, 10); $range = python\range(0, 10);
$type = python\type($value); $type = python\type($value);
``` ```
@ -445,8 +445,8 @@ $dict2 = python\dict();
- `$dict1``$dict2` 都是 `PyDict` typed object。 - `$dict1``$dict2` 都是 `PyDict` typed object。
- 两种写法必须使用相同的类型检查、方法解析和 Native Call 优化。 - 两种写法必须使用相同的类型检查、方法解析和 Native Call 优化。
- 语法糖不能退化成 `mixed`、`var` 或只有基础类型 `PyObject` - 语法糖不能退化成 `mixed`、`var` 或只有基础类型 `PyObject`
- Python builtin 调用同样遵守对象保持规则,例如 `python\len()` 返回包装 Python int 的 `PyObject`;需要先以 `toPlainValue()`(或兼容入口 `python\scalar()`)离开 Python 对象规则,再使用普通 TypePHP 转换得到确定类型。`python\print()` 的 Python `None` 结果也保持为 `PyObject`,作为独立语句使用时可直接丢弃。 - Python builtin 调用同样遵守对象保持规则,例如 `python\len()` 返回包装 Python int 的 `PyObject`;需要先以 `toValue()`(或函数入口 `python\scalar()`)离开 Python 对象规则,再使用普通 TypePHP 转换得到确定类型。`python\print()` 的 Python `None` 结果也保持为 `PyObject`,作为独立语句使用时可直接丢弃。
- `toPlainValue()``python\scalar()` 都不是普通 Python builtin 调用,而是明确要求退出 Python 类型规则的转换边界,因此返回 TypePHP `var` - `PyObject::toValue()``python\scalar()` 都不是普通 Python builtin 调用,而是明确要求退出 Python 类型规则的转换边界,因此返回 TypePHP `var`
- 动态 Python module 成员调用统一返回 `PyObject` - 动态 Python module 成员调用统一返回 `PyObject`
## 9. Python 对象类型 ## 9. Python 对象类型
@ -477,16 +477,16 @@ unset($object->name);
分别映射为 Python 的 `getattr`、call、`setattr` 和 `delattr` 协议。 分别映射为 Python 的 `getattr`、call、`setattr` 和 `delattr` 协议。
`PyObject` 与普通 Object 遵循相同的方法解析规则。TypePHP 不为它保留或注入 `toInt()`、`toFloat()`、`toBool()`、`toString()`、`toArray()` 等特殊转换方法;同名 Python 成员仍按正常的动态成员规则调用。 `PyObject` 明确提供 `toValue()``toArray()` 两个 PHP Facade 方法。`toValue()` 等价于 `PyCore::scalar()` / `python\scalar()`,把 Python 值递归转换为 PHP 内置值。其返回值再使用普通 TypePHP 转换方法确定类型:
`toPlainValue()` 是与 `toArray()`、`toString()` 同级的 TypePHP 全局关键词方法,用于把扩展对象转换为 PHP 内置值;当前第一个受支持的扩展对象是 `PyObject`。从 Python 对象进入 TypePHP 原生值时,推荐使用这个可保持链式调用的入口。`python\scalar()` 保留为等价的函数式入口。其返回值再使用普通 TypePHP 转换方法确定类型:
```php ```php
$pyValue = np\int64(42); // PyObject $pyValue = np\int64(42); // PyObject
$value = $pyValue->toPlainValue()->toInt(); // TypePHP int $value = $pyValue->toValue()->toInt(); // TypePHP int
``` ```
这里的 `toInt()` 作用于 `toPlainValue()` 已返回的 TypePHP 值,并非作用于 `PyObject` 这里的 `toInt()` 作用于 `toValue()` 已返回的 TypePHP 值,并非作用于 `PyObject`
`toArray()` 仅转换 Python `list`、`tuple`、`set`、`dict` 以及 iterator。容器元素递归转换为 PHP 值;iterator 会被消费,后续再次转换只能得到其剩余元素。不支持转换的 Python 类型返回空数组。`toArray()` 同时是 TypePHP 关键词方法,但 PHPX 的对象转换路径会调用 `PyObject::toArray()`;`toString()` 则继续通过关键词方法调用 `PyObject::__toString()`,phpy 不重复声明 `toString()`
### 10.2 下标 ### 10.2 下标
@ -630,7 +630,7 @@ TypePHP 的 Python 专用调用路径必须关闭 phpy 的返回值隐式转换
编译器已知的 phpy 构造语法糖仍保留精确子类,例如 `python\list()` 返回 `PyList`、`python\dict()` 返回 `PyDict`;这些类型本身都是 `PyObject` 子类,不构成返回值隐式转换。 编译器已知的 phpy 构造语法糖仍保留精确子类,例如 `python\list()` 返回 `PyList`、`python\dict()` 返回 `PyDict`;这些类型本身都是 `PyObject` 子类,不构成返回值隐式转换。
phpy Zend Facade 应提供相互独立的“保持 Python 对象”和“显式转换为 TypePHP”入口。不能通过修改进程级全局函数指针或全局转换模式来临时切换,否则嵌套调用、同步重入和异常路径可能把错误策略泄漏给后续调用。TypePHP 生成的普通 Python 调用只动态调用对象保持入口;`toPlainValue()` 与 `python\scalar()` 最终都调用明确的标量转换入口。 phpy Zend Facade 应提供相互独立的“保持 Python 对象”和“显式转换为 TypePHP”入口。不能通过修改进程级全局函数指针或全局转换模式来临时切换,否则嵌套调用、同步重入和异常路径可能把错误策略泄漏给后续调用。TypePHP 生成的普通 Python 调用只动态调用对象保持入口;`PyObject::toValue()` 与 `python\scalar()` 最终都调用明确的标量转换入口。
phpy 内部已使用 `PythonToPhpConverter``PhpToPythonConverter` 实现这一约束。每次顶层转换拥有独立实例,递归子值复用同一实例;容器进入与退出由 RAII guard 管理,循环容器和超过深度限制的输入会抛出 `PyError`,不会污染后续转换或导致进程崩溃。 phpy 内部已使用 `PythonToPhpConverter``PhpToPythonConverter` 实现这一约束。每次顶层转换拥有独立实例,递归子值复用同一实例;容器进入与退出由 RAII guard 管理,循环容器和超过深度限制的输入会抛出 `PyError`,不会污染后续转换或导致进程崩溃。
@ -644,28 +644,28 @@ phpy 内部已使用 `PythonToPhpConverter` 与 `PhpToPythonConverter` 实现这
### 14.2 显式转换 ### 14.2 显式转换
Python 对象只有通过 `toPlainValue()`、`python\scalar()`(或手写等价的 `PyCore::scalar()`)才能进入 TypePHP 类型规则: Python 对象只有通过 `toValue()`、`python\scalar()`(或手写等价的 `PyCore::scalar()`)才能进入 TypePHP 类型规则:
```php ```php
$nativeValue1 = PyCore::scalar($value); $nativeValue1 = PyCore::scalar($value);
$nativeValue2 = python\scalar($value); // 完全等价的语法糖 $nativeValue2 = python\scalar($value); // 完全等价的语法糖
$nativeValue3 = $value->toPlainValue(); // 推荐的链式关键词方法 $nativeValue3 = $value->toValue();
$integer = $value->toPlainValue()->toInt(); $integer = $value->toValue()->toInt();
$float = $value->toPlainValue()->toFloat(); $float = $value->toValue()->toFloat();
$boolean = $value->toPlainValue()->toBool(); $boolean = $value->toValue()->toBool();
$string = $value->toPlainValue()->toString(); $string = $value->toValue()->toString();
$array = $value->toPlainValue()->toArray(); $array = $value->toArray();
``` ```
规则: 规则:
- 编译器把 `toPlainValue()``python\scalar()` 识别为 Python/TypePHP 边界;其后的 `toInt()` 等调用是 TypePHP 原生值已有的普通转换能力 - `toValue()``PyObject` 的普通公开方法,不注册为 TypePHP 关键词方法;它在 phpy 内部复用与 `PyCore::scalar()` 相同的转换器
- `toPlainValue()` 注册在 TypePHP 全局 `KEYWORD_METHOD_MAP`,不属于 Python module 语法。它按 `toArray()` / `toString()` 的 PHPX 自由函数模式生成 `php::toPlainValue(value)`。参数可以是 `php::Object``php::Var`;当前 PHPX 在运行时确认对象是 `PyObject`,再通过 Zend API 调用 `PyCore::scalar()`,不链接 phpy C++ 符号。后续其他扩展类可继续在 PHPX 中增加适配 - `toArray()` 保留 TypePHP 全局关键词方法语义。PHPX 对对象执行数组转换时优先调用其公开的 `toArray()`,因此会进入 phpy 实现
- 显式转换完成后,结果完全进入 TypePHP 的静态类型、运算符和参数传递规则,不再采用 Python protocol。 - 显式转换完成后,结果完全进入 TypePHP 的静态类型、运算符和参数传递规则,不再采用 Python protocol。
- 容器转换属于显式深转换,并检测递归引用。 - 容器转换属于显式深转换,并检测递归引用。
- Python 大整数不能静默溢出;现有转换规则需要 review 后再确定与 TypePHP `BigInt` 的精确映射。 - Python 大整数不能静默溢出;现有转换规则需要 review 后再确定与 TypePHP `BigInt` 的精确映射。
- Python `str``bytes` 必须区分,不能都无条件转换为 TypePHP string。 - Python `str``bytes` 必须区分,不能都无条件转换为 TypePHP string。
- phpy 只负责 `PyCore::scalar()` 的 Python 到 PHP 值转换;后续 `toInt/toFloat/toBool/toString/toArray` 不属于 phpy,也不应在 `PyObject` 上重复实现 - phpy 负责 `PyObject::toValue()` / `PyCore::scalar()` 的通用值转换,以及受限的 `PyObject::toArray()` 容器转换。`toInt/toFloat/toBool` 属于转换后的 PHP 值;`toString()` 仍由 TypePHP 关键词方法调用 `PyObject::__toString()`
现有 phpy 的 PHP 用户仍可保留兼容行为;TypePHP 调用 phpy 的对象保持型 Zend API。为此可以重构或新增 phpy internal class method,但不增加 TypePHP 到 phpy 的 C++ 链接依赖。 现有 phpy 的 PHP 用户仍可保留兼容行为;TypePHP 调用 phpy 的对象保持型 Zend API。为此可以重构或新增 phpy internal class method,但不增加 TypePHP 到 phpy 的 C++ 链接依赖。
@ -714,7 +714,7 @@ identity 比较调用 `operator\is_()` / `operator\is_not()`。即使两个对
```php ```php
$result1 = $pyInt + 10; // 10 转为 Python int,由 Python 执行加法 $result1 = $pyInt + 10; // 10 转为 Python int,由 Python 执行加法
$result2 = $pyList * getCount(); // 先求值 getCount(),再转为 Python int $result2 = $pyList * getCount(); // 先求值 getCount(),再转为 Python int
$native = $pyInt->toPlainValue()->toInt() + 10; // 已显式转为 TypePHP int,使用 TypePHP 加法 $native = $pyInt->toValue()->toInt() + 10; // 已显式转为 TypePHP int,使用 TypePHP 加法
``` ```
`operator` 调用结果仍为 `PyObject`,以保留 Python 自定义运算符可能返回的任意对象。`===` / `!==` 和条件分支是例外:`operator.is_/is_not/truth()` 的 Python bool 结果随后通过显式 phpy 转换入口得到 TypePHP `bool`。两侧操作数必须严格从左到右各求值一次,转换过程不得导致表达式重复执行。 `operator` 调用结果仍为 `PyObject`,以保留 Python 自定义运算符可能返回的任意对象。`===` / `!==` 和条件分支是例外:`operator.is_/is_not/truth()` 的 Python bool 结果随后通过显式 phpy 转换入口得到 TypePHP `bool`。两侧操作数必须严格从左到右各求值一次,转换过程不得导致表达式重复执行。
@ -889,7 +889,7 @@ phpy 仓库现有 PHPUnit 用于验证 ZendVM/PHP Facade 与共享 Runtime:
- TypePHP 参数到 Python 的转换,以及 Python 返回值的显式转换。 - TypePHP 参数到 Python 的转换,以及 Python 返回值的显式转换。
- 空 TypePHP 数组默认转换为 Python list,以及数组递归深拷贝、异常中止和重复转换行为。 - 空 TypePHP 数组默认转换为 Python list,以及数组递归深拷贝、异常中止和重复转换行为。
- Python builtin、模块函数、方法和运算结果不会隐式变成 TypePHP 标量。 - Python builtin、模块函数、方法和运算结果不会隐式变成 TypePHP 标量。
- `$obj->toPlainValue()->toInt()`、`python\scalar($obj)->toInt()` 等显式边界及其后的普通 TypePHP 转换恢复静态类型和运算规则。 - `$obj->toValue()->toInt()`、`$obj->toArray()`、`python\scalar($obj)->toInt()` 等显式边界及其后的普通 TypePHP 转换恢复静态类型和运算规则。
- Python 异常到 TypePHP 异常。 - Python 异常到 TypePHP 异常。
- phpy 未加载时首次 Python 调用抛出 PHP `Error`,而仅声明未使用的 Python `use` 不报错。 - phpy 未加载时首次 Python 调用抛出 PHP `Error`,而仅声明未使用的 Python `use` 不报错。
- TypePHP callable 被 Python 回调。 - TypePHP callable 被 Python 回调。
@ -972,7 +972,7 @@ pytest 用于 phpy 自身已有 Python-facing bridge 的回归测试;它不表
16. `np\array()` 表示读取并调用 Python 包成员;该成员可以是函数、class 或其他 callable,具体类型由 Python 运行时决定。 16. `np\array()` 表示读取并调用 Python 包成员;该成员可以是函数、class 或其他 callable,具体类型由 Python 运行时决定。
17. `PyObject` 可以与 TypePHP 值混合运算;TypePHP 操作数转换为 Python 对象后,整个运算由 CPython protocol 执行,结果保持为 `PyObject` 17. `PyObject` 可以与 TypePHP 值混合运算;TypePHP 操作数转换为 Python 对象后,整个运算由 CPython protocol 执行,结果保持为 `PyObject`
18. Python 函数、方法、class 构造和 builtin 调用的结果一律保持为 `PyObject` 或已知的 phpy 子类;禁用 phpy 返回值隐式转换。 18. Python 函数、方法、class 构造和 builtin 调用的结果一律保持为 `PyObject` 或已知的 phpy 子类;禁用 phpy 返回值隐式转换。
19. `PyObject::toPlainValue()` 是推荐的链式显式转换关键词;`python\scalar()` 保留为等价入口。两者退出 Python 类型规则后均可继续使用普通 TypePHP 转换,例如 `$obj->toPlainValue()->toInt()` 19. `PyObject::toValue()` 是显式标量/容器转换方法,等价于 `python\scalar()`;`PyObject::toArray()` 只接受可转换容器和 iterator,不支持的类型返回空数组。转换后可继续使用普通 TypePHP 转换,例如 `$obj->toValue()->toInt()`
20. TypePHP 调用 Python 时,所有参数自动转换为 Python 类型;TypePHP 数组递归深拷贝,空数组默认转换为 Python list。 20. TypePHP 调用 Python 时,所有参数自动转换为 Python 类型;TypePHP 数组递归深拷贝,空数组默认转换为 Python list。
21. 性能敏感代码应复用 `PyDict`、`PyList`、`PyStr` 等代理对象,避免同一 TypePHP 值反复转换和深拷贝。 21. 性能敏感代码应复用 `PyDict`、`PyList`、`PyStr` 等代理对象,避免同一 TypePHP 值反复转换和深拷贝。
22. TypePHP 的主要语言增量是 `use python\...` 和模块别名;使用别名时通过与 `funcMap` 同类的 lazy indexed map 调用 phpy import,其他运行时能力优先直接复用 phpy。 22. TypePHP 的主要语言增量是 `use python\...` 和模块别名;使用别名时通过与 `funcMap` 同类的 lazy indexed map 调用 phpy import,其他运行时能力优先直接复用 phpy。

@ -32,11 +32,11 @@
1. TypePHP 参数从左到右求值后自动转换为 Python 值。 1. TypePHP 参数从左到右求值后自动转换为 Python 值。
2. 标量、数组、空数组、嵌套容器及 TypePHP callable 转换。 2. 标量、数组、空数组、嵌套容器及 TypePHP callable 转换。
3. 通过 `$py->toPlainValue()`兼容入口 `python\scalar($py)` 离开 Python 对象规则;需要确定原生类型时继续使用普通 TypePHP 转换,例如 `$py->toPlainValue()->toInt()` 3. 通过 `$py->toValue()``python\scalar($py)` 离开 Python 对象规则;需要确定原生类型时继续使用普通 TypePHP 转换,例如 `$py->toValue()->toInt()`。容器和 iterator 可直接使用 `$py->toArray()`
4. 深拷贝、递归容器、溢出、Unicode/bytes 和异常路径测试。 4. 深拷贝、递归容器、溢出、Unicode/bytes 和异常路径测试。
5. review 并重构 phpy 转换策略,移除影响同步重入的全局临时转换状态。 5. review 并重构 phpy 转换策略,移除影响同步重入的全局临时转换状态。
实现状态:核心边界已完成。TypePHP 参数严格从左到右求值,支持标量、空数组、嵌套 list/dict 与 callable;`toPlainValue()` 是推荐的链式显式转换入口,`python\scalar()` 保留为等价兼容入口。PHPX 在运行时确认 `php::Var` 确实持有 `PyObject`,再通过 Zend Facade 调用 `PyCore::scalar()`;不依赖 phpy C++ 符号。phpy 已移除进程级转换函数指针,改为局部有状态转换器、RAII 递归保护和 128 层深度限制,并覆盖无效 UTF-8、PHP 自引用数组及 Python 循环容器错误路径。Python 大整数与 bytes 的最终语言映射仍保留在本阶段后续工作中。 实现状态:核心边界已完成。TypePHP 参数严格从左到右求值,支持标量、空数组、嵌套 list/dict 与 callable;`PyObject::toValue()` 与 `python\scalar()` 复用 phpy 的显式转换入口,`PyObject::toArray()` 转换受支持的容器和 iterator。phpy 已移除进程级转换函数指针,改为局部有状态转换器、RAII 递归保护和 128 层深度限制,并覆盖无效 UTF-8、PHP 自引用数组及 Python 循环容器错误路径。Python 大整数与 bytes 的最终语言映射仍保留在本阶段后续工作中。
## 阶段 4:运算符 ## 阶段 4:运算符

@ -0,0 +1,86 @@
# Python 工具子模块
TypePHP 将 Python IDE helper 生成器和 Python 源码转换器集成到了 `tpc`。两者位于独立的
`src/PythonTools` 目录,只复用 `tpc` 命令入口,不进入正常的 PHP 预处理、C++ 生成和编译流水线。
## Python namespace IDE helper
```shell
./tpc --gen-python-helper math
./tpc --gen-python-helper numpy.linalg
./tpc --gen-python-helper numpy --output-dir .ide-helper
```
命令通过 PHPy 导入指定 Python module,并使用 Python `inspect` API 采集函数、参数、类、方法和
module attribute。PHPy 扩展以及目标 Python module 必须安装在执行 `tpc` 的主机环境中。
默认生成文件位于当前目录的 `ide-helper` 中。`--output-dir` 可以替换这个输出根目录,既支持
相对当前目录的路径,也支持绝对路径:
```text
ide-helper/python/math.php
ide-helper/python/numpy/linalg.php
ide-helper/PyObject.php
```
首次生成 module helper 时,会同时生成公共的 `PyObject.php`。它包含 `PyObject` 的动态访问、调用、
数组访问、迭代以及 `toArray()`、`toValue()` 等方法提示,供所有 Python module helper 共享。若该文件
已经存在,生成器会保留原文件,不进行覆盖。
生成内容使用 TypePHP 的 module-as-namespace 形式,例如 `python\math\sqrt()`,并兼容普通
`use`、`use function` 和 `use const` 的 IDE 名称解析。文件末尾包含 `die`,用于在误执行时明确
终止程序。helper 只能交给 IDE 索引,不能被 include,也不能加入 TypePHP 项目的 sources 或编译输入。
`PyObject::IDE_HELPER_ONLY` 是所有 helper 共用的提示常量。非 `void` stub 的方法体使用
`die(\PyObject::IDE_HELPER_ONLY)`,以满足 IDE 对返回类型控制流的检查,不会再产生“缺少 return
语句”的诊断。module attribute 使用命名空间 `const` 声明,支持 IDE 的常量补全和 `use const`
PHP 8.1 及以上允许在常量初始化表达式中使用 `new`。module attribute 因此直接使用仅供 IDE
分析的 `PyObject` 实例作为占位值:
```php
const pi = new \PyObject();
```
这样 IDE 会将常量精确识别为 `PyObject`,而不是从 `null` 推断出错误类型。
公共 `PyObject` helper 还声明了 TypePHP 的虚拟关键词方法,包括 `toInt()`、`toFloat()`、
`toString()`、`toBool()`、`toStream()`、高精度类型转换、`toObject()`、`toAny()` 和 `toRef()`
这些声明仅用于 IDE 补全;调用会在编译期展开,并不是 PHPy `PyObject` 运行时类的实体方法。
`toArray()``toValue()` 则仍是 PHPy 提供的真实方法。
Python class 的构造函数会显式调用 `parent::__construct()`。Python 对象若定义了 `count()`,helper
不会重复声明它,因为 `PyObject::count(): int` 已用于 PHP `Countable`。需要调用 Python 自身的
`count()` 时,应显式写为 `$object->__call('count', $arguments)`
PHP function/class 名称大小写不敏感,而 Python 名称大小写敏感;PHP 保留字也不能声明为普通
stub symbol。生成器会以注释报告无法用合法 PHP 声明表达的符号,不会擅自重命名 Python API。
## Python 转 TypePHP
```shell
./tpc --convert-python-to-php script.py > script.php
```
转换器调用 PATH 中的 `python3` 解析 Python AST,然后输出使用 TypePHP Python namespace
语法的 PHP 源码。普通 module import 会转换为 namespace import:
```python
import math
print(math.sqrt(16))
```
```php
use python\math;
function main(): void
{
python\print(math\sqrt(16));
}
```
当前支持普通 import、函数、赋值、调用、容器字面量、基础运算、单项比较、if/while/for、
lambda 和基础 f-string。module 顶层变量会转换为 PHP global,以保持函数读取 module 变量的能力。
转换器遵循“不能可靠保持语义就拒绝”的原则。class、async、generator、try/with、decorator、
destructuring assignment、chained comparison、嵌套函数以及 loop-else 等尚未完成的语法会抛出带
源文件和行号的错误,不会生成看似可用但语义错误的 PHP 代码。

@ -0,0 +1,13 @@
<?php
use const python\math\pi;
use function python\platform\python_version;
function main()
{
echo pi, "\n";
var_dump(get_class(pi));
var_dump(pi->toValue()->toFloat());
var_dump(pi->toValue()->toInt());
echo python_version(), "\n";
}

@ -0,0 +1,15 @@
<?php
function main(): void
{
$list = python\list([1, 2, 3]);
$array = $list->toArray();
$value = convertPythonValue($list);
$integer = python\int(42)->toValue()->toInt();
var_dump($array, $value, $integer);
}
function convertPythonValue(PyObject $value): mixed
{
return $value->toValue();
}

@ -1,10 +0,0 @@
<?php
function invalidPlainValueCall(PyObject $value): void
{
$value->toPlainValue(1);
}
function main(): void
{
}

@ -1,15 +0,0 @@
<?php
function main(): void
{
$list = python\list([1, 2, 3]);
$plain = $list->toPlainValue();
$dynamic = convertPlainValue($list);
$array = $dynamic->toArray();
var_dump($plain, $array);
}
function convertPlainValue(mixed $value): mixed
{
return $value->toPlainValue();
}

@ -953,7 +953,8 @@ YAML);
$cppFile = $this->compiler->convertFile($testFile); $cppFile = $this->compiler->convertFile($testFile);
$cpp = file_get_contents($cppFile); $cpp = file_get_contents($cppFile);
$this->assertStringContainsString('data = php::toArray(user);', $cpp); $this->assertStringContainsString('data = php_arrayableuser__toarray(user);', $cpp);
$this->assertStringNotContainsString('php::toArray(', $cpp);
$this->assertStringContainsString('php::Array php_arrayableuser__toarray(', $cpp); $this->assertStringContainsString('php::Array php_arrayableuser__toarray(', $cpp);
$this->assertStringContainsString('php::Str php_arrayableuser____tostring(', $cpp); $this->assertStringContainsString('php::Str php_arrayableuser____tostring(', $cpp);
} }

@ -293,33 +293,26 @@ final class PythonModuleTest extends TestCase
$this->assertStringNotContainsString('phpy::', $cpp); $this->assertStringNotContainsString('phpy::', $cpp);
} }
public function testToPlainValueUsesTheZendScalarFacade(): void public function testPyObjectConversionMethodsUseThePhpyFacade(): void
{ {
global $translator; global $translator;
$compiler = CompilerTest::create(ROOT_PATH); $compiler = CompilerTest::create(ROOT_PATH);
$translator = $compiler; $translator = $compiler;
$source = ROOT_PATH . '/phpunit/code/python/plain-value.php'; $source = ROOT_PATH . '/phpunit/code/python/object-conversion-methods.php';
$compiler->addFiles([$source]); $compiler->addFiles([$source]);
$compiler->prepareFile($source); $compiler->prepareFile($source);
$cpp = file_get_contents($compiler->convertFile($source)); $cpp = file_get_contents($compiler->convertFile($source));
$extension = file_get_contents($compiler->genExtension()); $extension = file_get_contents($compiler->genExtension());
$this->assertStringContainsString('php::Var plain;', $cpp); $this->assertStringContainsString('php::Array array;', $cpp);
$this->assertStringContainsString('php::Var dynamic;', $cpp); $this->assertStringContainsString('php::Var value;', $cpp);
$this->assertStringContainsString('php::toPlainValue(', $cpp); $this->assertStringNotContainsString('php::toArray(', $cpp);
$this->assertStringContainsString('php::Var value', $cpp); $this->assertStringContainsString('toArray', $extension);
$this->assertStringNotContainsString('.call("toPlainValue"', $cpp); $this->assertStringContainsString('toValue', $extension);
$this->assertStringNotContainsString('php::toPlainValue(', $cpp);
$this->assertStringNotContainsString('phpy::', $cpp); $this->assertStringNotContainsString('phpy::', $cpp);
} }
public function testToPlainValueRejectsArguments(): void
{
$this->expectException(TestError::class);
$this->expectExceptionMessage('The toPlainValue method does not accept parameters');
$this->compileFixture('plain-value-arguments.php');
}
private function compileFixture(string $file): void private function compileFixture(string $file): void
{ {
global $translator; global $translator;

@ -0,0 +1,156 @@
<?php
namespace TypePhpTest\PythonTools;
use PHPUnit\Framework\Attributes\RequiresPhpExtension;
use PHPUnit\Framework\TestCase;
use TypePhp\PythonTools\IdeHelper\HelperRenderer;
use TypePhp\PythonTools\IdeHelper\PhpyModuleScanner;
use TypePhp\PythonTools\IdeHelper\PyObjectHelperRenderer;
final class PythonIdeHelperTest extends TestCase
{
public function testRendererProducesInertNamespaceHelper(): void
{
$metadata = [
'module' => 'demo.widgets',
'doc' => 'Demo module.',
'attributes' => [
['name' => 'VERSION'],
['name' => 'class'],
],
'functions' => [
[
'name' => 'create',
'parameters' => [
['name' => 'value', 'optional' => false, 'variadic' => false],
['name' => 'mode', 'optional' => true, 'variadic' => false],
],
],
],
'classes' => [
[
'name' => 'Widget',
'parameters' => [],
'methods' => [
['name' => 'render', 'parameters' => []],
],
'properties' => ['name'],
],
],
];
$helper = (new HelperRenderer())->render($metadata);
self::assertStringContainsString('@generated TypePHP Python IDE helper', $helper);
self::assertStringContainsString('namespace python\\demo\\widgets;', $helper);
self::assertStringNotContainsString('if (false)', $helper);
self::assertStringEndsWith("die(\\PyObject::IDE_HELPER_ONLY);\n", $helper);
self::assertStringContainsString('const VERSION = new \\PyObject();', $helper);
self::assertStringContainsString("\nconst VERSION = new \\PyObject();", $helper);
self::assertStringNotContainsString('/** @var \\PyObject */', $helper);
self::assertStringContainsString("\nclass Widget extends \\PyObject", $helper);
self::assertStringContainsString("\n public function render()", $helper);
self::assertStringNotContainsString('const class =', $helper);
self::assertStringNotContainsString('define(', $helper);
self::assertStringContainsString(
'function create(mixed $value, mixed $mode = null): \\PyObject { die(\\PyObject::IDE_HELPER_ONLY); }',
$helper,
);
self::assertStringContainsString('class Widget extends \\PyObject', $helper);
self::assertStringContainsString('function Widget(): Widget { die(\\PyObject::IDE_HELPER_ONLY); }', $helper);
self::assertStringContainsString('public function __construct() { parent::__construct(); }', $helper);
self::assertStringContainsString('public function render(): \\PyObject { die(\\PyObject::IDE_HELPER_ONLY); }', $helper);
self::assertStringNotContainsString('PyCore::import', $helper);
}
public function testBuiltinsAreRenderedInPythonRootNamespace(): void
{
$helper = (new HelperRenderer())->render([
'module' => 'builtins',
'doc' => '',
'attributes' => [],
'functions' => [['name' => 'len', 'parameters' => []]],
'classes' => [],
]);
self::assertStringContainsString('namespace python;', $helper);
self::assertStringContainsString('function len(): \\PyObject', $helper);
}
public function testRendererProducesAnInertPyObjectHelper(): void
{
$helper = (new PyObjectHelperRenderer())->render();
self::assertStringNotContainsString('if (false)', $helper);
self::assertStringEndsWith("die(PyObject::IDE_HELPER_ONLY);\n", $helper);
self::assertStringContainsString(
'class PyObject implements \\ArrayAccess, \\Iterator, \\Countable',
$helper,
);
self::assertStringContainsString("\nclass PyObject implements", $helper);
self::assertStringContainsString("\n public const IDE_HELPER_ONLY", $helper);
self::assertStringContainsString("public const IDE_HELPER_ONLY = 'IDE helper only';", $helper);
self::assertStringNotContainsString('enum PyObjectConstant', $helper);
self::assertStringContainsString('public function toArray(): array { die(self::IDE_HELPER_ONLY); }', $helper);
self::assertStringContainsString('public function toValue(): mixed { die(self::IDE_HELPER_ONLY); }', $helper);
self::assertStringContainsString('public function __toString(): string { die(self::IDE_HELPER_ONLY); }', $helper);
self::assertStringContainsString('public function next(): void {}', $helper);
self::assertStringContainsString('TypePHP keyword methods are compiler intrinsics', $helper);
self::assertStringContainsString('public function toInt(): int { die(self::IDE_HELPER_ONLY); }', $helper);
self::assertStringContainsString('public function toFloat(): float { die(self::IDE_HELPER_ONLY); }', $helper);
self::assertStringContainsString('public function toString(): string { die(self::IDE_HELPER_ONLY); }', $helper);
self::assertStringContainsString('public function toBool(): bool { die(self::IDE_HELPER_ONLY); }', $helper);
self::assertStringContainsString('public function toStream(): mixed { die(self::IDE_HELPER_ONLY); }', $helper);
self::assertStringContainsString('public function toBigInt(): mixed { die(self::IDE_HELPER_ONLY); }', $helper);
self::assertStringContainsString('public function toBigFloat(): mixed { die(self::IDE_HELPER_ONLY); }', $helper);
self::assertStringContainsString('public function toDecimal(): mixed { die(self::IDE_HELPER_ONLY); }', $helper);
self::assertStringContainsString('public function toObject(?string $class = null): object', $helper);
self::assertStringContainsString('public function toAny(): mixed { die(self::IDE_HELPER_ONLY); }', $helper);
self::assertStringContainsString('public function toRef(): mixed { die(self::IDE_HELPER_ONLY); }', $helper);
self::assertSame(1, substr_count($helper, 'public function toArray(): array'));
}
public function testRendererOmitsPythonCountMethodInheritedFromPyObject(): void
{
$helper = (new HelperRenderer())->render([
'module' => 'demo',
'attributes' => [],
'functions' => [],
'classes' => [[
'name' => 'Container',
'parameters' => [],
'properties' => [],
'methods' => [
['name' => 'count', 'parameters' => [['name' => 'value']]],
['name' => 'append', 'parameters' => [['name' => 'value']]],
],
]],
]);
self::assertStringNotContainsString('public function count(', $helper);
self::assertStringContainsString('public function append(mixed $value): \\PyObject', $helper);
}
#[RequiresPhpExtension('phpy')]
public function testPhpyScannerReadsRealPythonModule(): void
{
$metadata = (new PhpyModuleScanner())->scan('math');
$functions = array_column($metadata['functions'], null, 'name');
$attributes = array_column($metadata['attributes'], null, 'name');
self::assertArrayHasKey('sqrt', $functions);
self::assertSame('x', $functions['sqrt']['parameters'][0]['name']);
self::assertArrayHasKey('pi', $attributes);
}
#[RequiresPhpExtension('phpy')]
public function testPhpyScannerReadsClassesWithoutRetainingDynamicCallTrampolines(): void
{
$metadata = (new PhpyModuleScanner())->scan('json');
$classes = array_column($metadata['classes'], null, 'name');
self::assertArrayHasKey('JSONDecoder', $classes);
self::assertNotEmpty($classes['JSONDecoder']['methods']);
}
}

@ -0,0 +1,76 @@
<?php
namespace TypePhpTest\PythonTools;
use PHPUnit\Framework\TestCase;
use RuntimeException;
use TypePhp\PythonTools\Converter\PythonToTypePhpConverter;
final class PythonToTypePhpConverterTest extends TestCase
{
public function testConvertsImportsFunctionsAndTopLevelCode(): void
{
$source = <<<'PYTHON'
import math
import os.path as path
from json import dumps as encode
def hypotenuse(x, y=4):
return math.sqrt(x * x + y * y)
value = hypotenuse(3)
print(encode({"value": value}))
PYTHON;
$php = (new PythonToTypePhpConverter())->convertSource($source, 'example.py');
self::assertStringContainsString('use python\\math;', $php);
self::assertStringContainsString('use python\\os\\path;', $php);
self::assertStringContainsString('function hypotenuse($x, $y = 4)', $php);
self::assertStringContainsString('return math\\sqrt($x * $x + $y * $y);', $php);
self::assertStringContainsString('function main(): void', $php);
self::assertStringContainsString('python\\json\\dumps(python\\dict([\'value\' => $value]))', $php);
self::assertStringContainsString('python\\print(', $php);
}
public function testPassDoesNotBecomeReturnAndPythonComparisonsStayExplicit(): void
{
$source = <<<'PYTHON'
def inspect_value(value, values):
if value is None:
pass
return value in values
PYTHON;
$php = (new PythonToTypePhpConverter())->convertSource($source, 'comparison.py');
self::assertStringContainsString('if ($value === null)', $php);
self::assertStringContainsString('// pass', $php);
self::assertStringContainsString('python\\operator\\contains($values, $value)', $php);
self::assertStringNotContainsString('if ($value === null) {' . "\n return;", $php);
}
public function testUnsupportedSyntaxReportsSourceLocation(): void
{
$this->expectException(RuntimeException::class);
$this->expectExceptionMessage('sample.py:1');
$this->expectExceptionMessage('ClassDef');
(new PythonToTypePhpConverter())->convertSource("class Demo:\n pass\n", 'sample.py');
}
public function testModuleVariablesRemainVisibleInsideFunctions(): void
{
$php = (new PythonToTypePhpConverter())->convertSource(<<<'PYTHON'
factor = 4
def scale(value):
return value * factor
print(scale(3))
PYTHON, 'globals.py');
self::assertStringContainsString('function scale($value)' . "\n{\n" . ' global $factor;', $php);
self::assertStringContainsString("function main(): void\n{\n" . ' global $factor;', $php);
}
}

@ -0,0 +1,36 @@
<?php
namespace TypePhpTest\PythonTools;
use PHPUnit\Framework\Attributes\RequiresPhpExtension;
use PHPUnit\Framework\TestCase;
use TypePhp\PythonTools\Command;
final class PythonToolsCommandTest extends TestCase
{
#[RequiresPhpExtension('phpy')]
public function testCustomOutputDirectoryPreservesExistingPyObjectHelper(): void
{
$root = sys_get_temp_dir() . '/typephp-python-tools-' . bin2hex(random_bytes(6));
$output = $root . '/stubs';
self::assertTrue(mkdir($output, 0777, true));
self::assertNotFalse(file_put_contents($output . '/PyObject.php', 'keep-me'));
try {
$status = Command::execute(
['tpc', Command::GENERATE_HELPER, 'math', '--output-dir', 'stubs'],
$root,
);
self::assertSame(0, $status);
self::assertFileExists($output . '/python/math.php');
self::assertSame('keep-me', file_get_contents($output . '/PyObject.php'));
} finally {
@unlink($output . '/python/math.php');
@rmdir($output . '/python');
@unlink($output . '/PyObject.php');
@rmdir($output);
@rmdir($root);
}
}
}

@ -167,7 +167,6 @@ class CompilerBase implements PropertyAccessContext
'toDecimal' => Type::DECIMAL, 'toDecimal' => Type::DECIMAL,
'toObject' => Type::OBJECT, 'toObject' => Type::OBJECT,
'toAny' => Type::VAR, 'toAny' => Type::VAR,
'toPlainValue' => Type::VAR,
'toRef' => Type::REF, 'toRef' => Type::REF,
]; ];

@ -350,7 +350,18 @@ trait MethodCallTrait
if ($methodName === 'toRef') { if ($methodName === 'toRef') {
return $this->genToRefCall($expr); return $this->genToRefCall($expr);
} }
return $this->genToConvertCall($object, $methodName, $receiverType); $receiverClass = $class;
if ($receiverClass === '' && !$this->isVarExpr($expr->var)) {
$receiverClass = $this->detectClassOfExpr($expr->var);
}
// A statically known object method preserves keyword priority
// while avoiding the generic PHPX conversion helper.
$useDeclaredToArray = $methodName === 'toArray'
&& $receiverClass !== ''
&& $this->objectTypeDeclaresMethod($receiverClass, $methodName);
if (!$useDeclaredToArray) {
return $this->genToConvertCall($object, $methodName, $receiverType);
}
} }
// MethodsFor('*') extensions apply to every receiver type. // MethodsFor('*') extensions apply to every receiver type.
$kwExt = $this->findKeywordExtensionMethod($methodName); $kwExt = $this->findKeywordExtensionMethod($methodName);

@ -452,7 +452,6 @@ trait UniversalMethodCall
'toBigInt' => 'php::BigInt::newInstance(' . $receiver . ')', 'toBigInt' => 'php::BigInt::newInstance(' . $receiver . ')',
'toBigFloat' => 'php::BigFloat::newInstance(' . $receiver . ')', 'toBigFloat' => 'php::BigFloat::newInstance(' . $receiver . ')',
'toDecimal' => 'php::Decimal::newInstance(' . $receiver . ')', 'toDecimal' => 'php::Decimal::newInstance(' . $receiver . ')',
'toPlainValue' => 'php::toPlainValue(' . $receiver . ')',
'toAny' => $receiver, 'toAny' => $receiver,
default => $receiver, default => $receiver,
}; };

@ -0,0 +1,145 @@
<?php
namespace TypePhp\PythonTools;
use RuntimeException;
use Throwable;
use TypePhp\PythonTools\Converter\PythonToTypePhpConverter;
use TypePhp\PythonTools\IdeHelper\HelperRenderer;
use TypePhp\PythonTools\IdeHelper\PhpyModuleScanner;
use TypePhp\PythonTools\IdeHelper\PyObjectHelperRenderer;
final class Command
{
public const GENERATE_HELPER = '--gen-python-helper';
public const CONVERT_SOURCE = '--convert-python-to-php';
/** Return null for a normal compiler invocation, otherwise an exit status. */
public static function execute(array $argv, ?string $workingDirectory = null): ?int
{
$helperIndex = array_search(self::GENERATE_HELPER, $argv, true);
$converterIndex = array_search(self::CONVERT_SOURCE, $argv, true);
if ($helperIndex === false && $converterIndex === false) {
return null;
}
if ($helperIndex !== false && $converterIndex !== false) {
return self::error('Python tool subcommands cannot be combined');
}
try {
if ($helperIndex !== false) {
$root = $workingDirectory ?? getcwd();
if (!is_string($root) || $root === '') {
throw new RuntimeException('Unable to determine the working directory');
}
[$module, $outputDirectory] = self::helperArguments($argv, $helperIndex, $root);
$metadata = (new PhpyModuleScanner())->scan($module);
$code = (new HelperRenderer())->render($metadata);
$relative = str_replace('.', DIRECTORY_SEPARATOR, $module) . '.php';
$file = $outputDirectory . DIRECTORY_SEPARATOR . 'python' . DIRECTORY_SEPARATOR . $relative;
$pyObjectFile = $outputDirectory . DIRECTORY_SEPARATOR . 'PyObject.php';
if (!is_file($pyObjectFile)) {
self::writeFile($pyObjectFile, (new PyObjectHelperRenderer())->render());
fwrite(STDOUT, "Generated PyObject IDE helper: {$pyObjectFile}" . PHP_EOL);
}
self::writeFile($file, $code);
fwrite(STDOUT, "Generated Python IDE helper: {$file}" . PHP_EOL);
return 0;
}
$file = self::singleArgument($argv, $converterIndex, self::CONVERT_SOURCE, '[your_file.py]');
fwrite(STDOUT, (new PythonToTypePhpConverter())->convertFile($file));
return 0;
} catch (Throwable $exception) {
return self::error($exception->getMessage());
}
}
/** @return array{string, string} */
private static function helperArguments(array $argv, int $optionIndex, string $root): array
{
$module = $argv[$optionIndex + 1] ?? '';
if (!is_string($module) || $module === '' || str_starts_with($module, '-')) {
throw new RuntimeException("Usage: {$argv[0]} " . self::GENERATE_HELPER
. ' [Python Module] [--output-dir <directory>]');
}
$output = null;
for ($index = 1, $count = count($argv); $index < $count; $index++) {
$argument = $argv[$index];
if ($index === $optionIndex || $index === $optionIndex + 1) {
continue;
}
if ($argument === '--output-dir') {
if ($output !== null || !isset($argv[$index + 1]) || $argv[$index + 1] === '') {
throw new RuntimeException('--output-dir requires exactly one directory');
}
$output = $argv[++$index];
continue;
}
if (str_starts_with($argument, '--output-dir=')) {
if ($output !== null) {
throw new RuntimeException('--output-dir may only be specified once');
}
$output = substr($argument, strlen('--output-dir='));
if ($output === '') {
throw new RuntimeException('--output-dir requires exactly one directory');
}
continue;
}
throw new RuntimeException("Unknown argument for " . self::GENERATE_HELPER . ": {$argument}");
}
if ($output === null) {
return [$module, $root . DIRECTORY_SEPARATOR . 'ide-helper'];
}
if (self::isAbsolutePath($output)) {
$absoluteOutput = rtrim($output, '/\\');
return [$module, $absoluteOutput === '' ? DIRECTORY_SEPARATOR : $absoluteOutput];
}
return [$module, $root . DIRECTORY_SEPARATOR . rtrim($output, '/\\')];
}
private static function isAbsolutePath(string $path): bool
{
return str_starts_with($path, '/')
|| str_starts_with($path, '\\')
|| preg_match('/^[A-Za-z]:[\\\\\/]/D', $path) === 1;
}
private static function singleArgument(array $argv, int $optionIndex, string $option, string $placeholder): string
{
$arguments = [];
foreach (array_slice($argv, 1) as $argument) {
if ($argument !== $option) {
$arguments[] = $argument;
}
}
if (count($arguments) !== 1 || $arguments[0] === '' || str_starts_with($arguments[0], '-')) {
throw new RuntimeException("Usage: {$argv[0]} {$option} {$placeholder}");
}
if (!isset($argv[$optionIndex + 1]) || $argv[$optionIndex + 1] !== $arguments[0]) {
throw new RuntimeException("{$option} must be followed by {$placeholder}");
}
return $arguments[0];
}
private static function writeFile(string $file, string $contents): void
{
$directory = dirname($file);
if (!is_dir($directory) && !mkdir($directory, 0777, true) && !is_dir($directory)) {
throw new RuntimeException("Unable to create IDE helper directory: {$directory}");
}
$temporary = $file . '.tmp-' . bin2hex(random_bytes(4));
if (file_put_contents($temporary, $contents) === false || !rename($temporary, $file)) {
@unlink($temporary);
throw new RuntimeException("Unable to write IDE helper: {$file}");
}
}
private static function error(string $message): int
{
fwrite(STDERR, "\033[31mError: {$message}\033[0m" . PHP_EOL);
return 1;
}
}

@ -0,0 +1,91 @@
<?php
namespace TypePhp\PythonTools\Converter;
use RuntimeException;
final class PythonAstLoader
{
private const DUMPER = <<<'PYTHON'
import ast
import json
import sys
def convert(value):
if isinstance(value, ast.AST):
result = {name: convert(item) for name, item in ast.iter_fields(value)}
result['_type'] = value.__class__.__name__
for name in ('lineno', 'col_offset', 'end_lineno', 'end_col_offset'):
if hasattr(value, name):
result[name] = getattr(value, name)
return result
if isinstance(value, list):
return [convert(item) for item in value]
if isinstance(value, bytes):
return {'_python_constant': 'bytes', 'hex': value.hex()}
if isinstance(value, complex):
return {'_python_constant': 'complex', 'real': value.real, 'imag': value.imag}
return value
filename = sys.argv[1]
source = sys.stdin.read()
try:
tree = ast.parse(source, filename=filename, type_comments=True)
except SyntaxError as error:
print(json.dumps({
'error': error.msg,
'line': error.lineno,
'column': error.offset,
}), file=sys.stderr)
raise SystemExit(2)
print(json.dumps(convert(tree), ensure_ascii=False))
PYTHON;
public function __construct(private readonly string $python = 'python3')
{
}
/** @return array<string, mixed> */
public function parse(string $source, string $filename): array
{
$command = [$this->python, '-c', self::DUMPER, $filename];
$descriptors = [
0 => ['pipe', 'r'],
1 => ['pipe', 'w'],
2 => ['pipe', 'w'],
];
// Do not inherit the project directory as Python's import root: a user
// file such as ast.py must not shadow the standard-library ast module.
$process = proc_open($command, $descriptors, $pipes, sys_get_temp_dir());
if (!is_resource($process)) {
throw new RuntimeException("Unable to start Python executable `{$this->python}`");
}
fwrite($pipes[0], $source);
fclose($pipes[0]);
$stdout = stream_get_contents($pipes[1]);
fclose($pipes[1]);
$stderr = stream_get_contents($pipes[2]);
fclose($pipes[2]);
$status = proc_close($process);
if ($status !== 0) {
$detail = trim($stderr);
try {
$error = json_decode($detail, true, flags: JSON_THROW_ON_ERROR);
if (is_array($error) && isset($error['error'])) {
$detail = $filename . ':' . ($error['line'] ?? 0) . ': ' . $error['error'];
}
} catch (\JsonException) {
}
throw new RuntimeException('Unable to parse Python source: ' . ($detail !== '' ? $detail : "exit status {$status}"));
}
try {
$tree = json_decode($stdout, true, flags: JSON_THROW_ON_ERROR);
} catch (\JsonException $exception) {
throw new RuntimeException('Python AST dumper returned invalid JSON', 0, $exception);
}
if (!is_array($tree) || ($tree['_type'] ?? null) !== 'Module') {
throw new RuntimeException('Python AST dumper did not return a module');
}
return $tree;
}
}

@ -0,0 +1,648 @@
<?php
namespace TypePhp\PythonTools\Converter;
use RuntimeException;
final class PythonToTypePhpConverter
{
/** @var array<string, string> */
private array $moduleAliases = [];
/** @var array<string, array{module: string, member: string}> */
private array $importedSymbols = [];
/** @var array<string, true> */
private array $definedFunctions = [];
/** @var array<string, true> */
private array $moduleGlobals = [];
private string $filename = '<python>';
private int $indent = 0;
public function __construct(private readonly PythonAstLoader $loader = new PythonAstLoader())
{
}
public function convertFile(string $file): string
{
$source = @file_get_contents($file);
if ($source === false) {
throw new RuntimeException("Unable to read Python source file: {$file}");
}
return $this->convertSource($source, $file);
}
public function convertSource(string $source, string $filename = '<python>'): string
{
$this->filename = $filename;
$this->moduleAliases = [];
$this->importedSymbols = [];
$this->definedFunctions = [];
$this->moduleGlobals = [];
$this->indent = 0;
$tree = $this->loader->parse($source, $filename);
$functions = [];
$main = [];
foreach ($tree['body'] ?? [] as $node) {
if (in_array($node['_type'] ?? '', ['Assign', 'AnnAssign', 'AugAssign'], true)) {
$targets = ($node['_type'] ?? '') === 'Assign' ? ($node['targets'] ?? []) : [$node['target'] ?? []];
foreach ($targets as $target) {
if (($target['_type'] ?? '') === 'Name') {
$this->moduleGlobals[(string) $target['id']] = true;
}
}
}
$type = $node['_type'] ?? '';
if ($type === 'Import' || $type === 'ImportFrom') {
$this->collectImport($node);
} elseif ($type === 'FunctionDef') {
$name = (string) $node['name'];
if ($name === 'main') {
$this->unsupported($node, 'a Python function named main conflicts with the TypePHP entry point');
}
$this->definedFunctions[$name] = true;
$functions[] = $node;
} else {
$main[] = $node;
}
}
$lines = ['<?php', '', '/** @generated from ' . $this->safeComment($filename) . ' */'];
foreach ($this->moduleAliases as $alias => $module) {
$namespace = 'python\\' . str_replace('.', '\\', $module);
$defaultAlias = str_replace('.', '\\', $module);
$lines[] = $alias === basename(str_replace('.', '/', $module))
? 'use ' . $namespace . ';'
: 'use ' . $namespace . ' as ' . $alias . ';';
}
if ($this->moduleAliases !== []) {
$lines[] = '';
}
foreach ($functions as $function) {
array_push($lines, ...$this->statement($function));
$lines[] = '';
}
$lines[] = 'function main(): void';
$lines[] = '{';
$this->indent = 1;
if ($this->moduleGlobals !== []) {
$lines[] = $this->line('global ' . implode(', ', $this->variables(array_keys($this->moduleGlobals))) . ';');
}
foreach ($main as $node) {
array_push($lines, ...$this->statement($node));
}
$this->indent = 0;
$lines[] = '}';
$lines[] = '';
return implode(PHP_EOL, $lines);
}
/** @param array<string, mixed> $node */
private function collectImport(array $node): void
{
if ($node['_type'] === 'Import') {
foreach ($node['names'] ?? [] as $name) {
$module = (string) $name['name'];
$alias = (string) ($name['asname'] ?? '');
if ($alias === '') {
$alias = explode('.', $module)[0];
$module = $alias;
}
$this->moduleAliases[$alias] = $module;
}
return;
}
if (($node['level'] ?? 0) !== 0 || ($node['module'] ?? null) === null) {
$this->unsupported($node, 'relative imports are not supported yet');
}
foreach ($node['names'] ?? [] as $name) {
if (($name['name'] ?? '') === '*') {
$this->unsupported($node, 'star imports are not supported');
}
$alias = (string) (($name['asname'] ?? null) ?: $name['name']);
$this->importedSymbols[$alias] = [
'module' => (string) $node['module'],
'member' => (string) $name['name'],
];
}
}
/** @param array<string, mixed> $node @return list<string> */
private function statement(array $node): array
{
$type = $node['_type'] ?? '';
return match ($type) {
'FunctionDef' => $this->functionDefinition($node),
'Assign' => $this->assignment($node),
'AnnAssign' => ($node['value'] ?? null) === null
? $this->unsupported($node, 'annotation-only assignments have no TypePHP runtime value')
: [$this->line($this->target($node['target']) . ' = ' . $this->expression($node['value']) . ';')],
'AugAssign' => [$this->line($this->target($node['target']) . ' ' . $this->binaryOperator($node['op'], $node)
. '= ' . $this->expression($node['value']) . ';')],
'Expr' => $this->expressionStatement($node),
'Return' => [$this->line('return' . (($node['value'] ?? null) === null ? '' : ' ' . $this->expression($node['value'])) . ';')],
'If' => $this->ifStatement($node),
'While' => $this->whileStatement($node),
'For' => $this->forStatement($node),
'Break' => [$this->line('break;')],
'Continue' => [$this->line('continue;')],
'Pass' => [$this->line('// pass')],
'Global' => [$this->line('global ' . implode(', ', $this->variables($node['names'] ?? [])) . ';')],
'Delete' => $this->deleteStatement($node),
'Import', 'ImportFrom' => [],
default => $this->unsupported($node),
};
}
/** @param array<string, mixed> $node @return list<string> */
private function functionDefinition(array $node): array
{
if ($this->indent !== 0) {
$this->unsupported($node, 'nested functions require Python closure scope analysis');
}
if (($node['decorator_list'] ?? []) !== []) {
$this->unsupported($node, 'function decorators are not supported yet');
}
$parameters = $this->parameters($node['args'], $node);
$lines = [$this->line('function ' . $node['name'] . '(' . $parameters . ')'), $this->line('{')];
$this->indent++;
$locals = $this->functionLocalNames($node);
$globals = array_values(array_diff(array_keys($this->moduleGlobals), array_keys($locals)));
if ($globals !== []) {
$lines[] = $this->line('global ' . implode(', ', $this->variables($globals)) . ';');
}
foreach ($node['body'] ?? [] as $body) {
array_push($lines, ...$this->statement($body));
}
$this->indent--;
$lines[] = $this->line('}');
return $lines;
}
/** @param array<string, mixed> $arguments @param array<string, mixed> $owner */
private function parameters(array $arguments, array $owner): string
{
$positional = array_merge($arguments['posonlyargs'] ?? [], $arguments['args'] ?? []);
$defaults = $arguments['defaults'] ?? [];
$defaultStart = count($positional) - count($defaults);
$result = [];
foreach ($positional as $index => $argument) {
$value = $this->variable((string) $argument['arg']);
if ($index >= $defaultStart) {
$value .= ' = ' . $this->expression($defaults[$index - $defaultStart]);
}
$result[] = $value;
}
foreach ($arguments['kwonlyargs'] ?? [] as $index => $argument) {
$default = $arguments['kw_defaults'][$index] ?? null;
$result[] = $this->variable((string) $argument['arg']) . ' = '
. ($default === null ? 'null' : $this->expression($default));
}
$variadic = $arguments['vararg'] ?? $arguments['kwarg'] ?? null;
if ($variadic !== null) {
$result[] = '...' . $this->variable((string) $variadic['arg']);
}
if (($arguments['vararg'] ?? null) !== null && ($arguments['kwarg'] ?? null) !== null) {
$this->unsupported($owner, 'simultaneous *args and **kwargs cannot be represented by one PHP signature');
}
return implode(', ', $result);
}
/** @param array<string, mixed> $node @return list<string> */
private function assignment(array $node): array
{
if (count($node['targets'] ?? []) !== 1) {
$this->unsupported($node, 'chained assignments are not supported yet');
}
$target = $node['targets'][0];
if (in_array($target['_type'] ?? '', ['Tuple', 'List'], true)) {
$this->unsupported($node, 'destructuring assignments are not supported yet');
}
return [$this->line($this->target($target) . ' = ' . $this->expression($node['value']) . ';')];
}
/** @param array<string, mixed> $node @return list<string> */
private function expressionStatement(array $node): array
{
$value = $node['value'];
if (($value['_type'] ?? '') === 'Constant' && is_string($value['value'] ?? null)) {
return [$this->line('/** ' . $this->safeComment($value['value']) . ' */')];
}
return [$this->line($this->expression($value) . ';')];
}
/** @param array<string, mixed> $node @return list<string> */
private function ifStatement(array $node, bool $elseif = false): array
{
$lines = [$this->line(($elseif ? 'elseif' : 'if') . ' (' . $this->expression($node['test']) . ')'), $this->line('{')];
$this->indent++;
foreach ($node['body'] ?? [] as $body) {
array_push($lines, ...$this->statement($body));
}
$this->indent--;
$lines[] = $this->line('}');
$otherwise = $node['orelse'] ?? [];
if (count($otherwise) === 1 && ($otherwise[0]['_type'] ?? '') === 'If') {
$nested = $this->ifStatement($otherwise[0], true);
$nested[0] = $this->line('elseif (' . $this->expression($otherwise[0]['test']) . ')');
array_push($lines, ...$nested);
} elseif ($otherwise !== []) {
$lines[] = $this->line('else');
$lines[] = $this->line('{');
$this->indent++;
foreach ($otherwise as $body) {
array_push($lines, ...$this->statement($body));
}
$this->indent--;
$lines[] = $this->line('}');
}
return $lines;
}
/** @param array<string, mixed> $node @return list<string> */
private function whileStatement(array $node): array
{
if (($node['orelse'] ?? []) !== []) {
$this->unsupported($node, 'while/else is not supported yet');
}
$lines = [$this->line('while (' . $this->expression($node['test']) . ')'), $this->line('{')];
$this->indent++;
foreach ($node['body'] ?? [] as $body) {
array_push($lines, ...$this->statement($body));
}
$this->indent--;
$lines[] = $this->line('}');
return $lines;
}
/** @param array<string, mixed> $node @return list<string> */
private function forStatement(array $node): array
{
if (($node['orelse'] ?? []) !== []) {
$this->unsupported($node, 'for/else is not supported yet');
}
if (($node['target']['_type'] ?? '') !== 'Name') {
$this->unsupported($node, 'only a simple for-loop target is supported');
}
$lines = [$this->line('foreach (' . $this->expression($node['iter']) . ' as '
. $this->variable($node['target']['id']) . ')'), $this->line('{')];
$this->indent++;
foreach ($node['body'] ?? [] as $body) {
array_push($lines, ...$this->statement($body));
}
$this->indent--;
$lines[] = $this->line('}');
return $lines;
}
/** @param array<string, mixed> $node @return list<string> */
private function deleteStatement(array $node): array
{
$lines = [];
foreach ($node['targets'] ?? [] as $target) {
if (!in_array($target['_type'] ?? '', ['Name', 'Attribute', 'Subscript'], true)) {
$this->unsupported($node, 'unsupported del target');
}
$lines[] = $this->line('unset(' . $this->target($target) . ');');
}
return $lines;
}
/** @param array<string, mixed> $node */
private function expression(array $node): string
{
return match ($node['_type'] ?? '') {
'Constant' => $this->constant($node['value'] ?? null),
'Name' => $this->nameExpression((string) $node['id']),
'Attribute' => $this->attribute($node),
'Call' => $this->call($node),
'List' => 'python\\list([' . $this->expressionList($node['elts'] ?? []) . '])',
'Tuple' => 'python\\tuple([' . $this->expressionList($node['elts'] ?? []) . '])',
'Set' => 'python\\set([' . $this->expressionList($node['elts'] ?? []) . '])',
'Dict' => 'python\\dict([' . $this->dictionaryItems($node) . '])',
'BinOp' => $this->binaryExpression($node),
'UnaryOp' => $this->unaryExpression($node),
'Compare' => $this->comparison($node),
'IfExp' => '(' . $this->expression($node['test']) . ' ? ' . $this->expression($node['body'])
. ' : ' . $this->expression($node['orelse']) . ')',
'Subscript' => $this->expression($node['value']) . '[' . $this->slice($node['slice']) . ']',
'Lambda' => 'fn (' . $this->parameters($node['args'], $node) . ') => ' . $this->expression($node['body']),
'JoinedStr' => $this->joinedString($node),
'Starred' => '...' . $this->expression($node['value']),
default => $this->unsupported($node),
};
}
/** @param array<string, mixed> $node */
private function call(array $node): string
{
$function = $node['func'];
if (($function['_type'] ?? '') === 'Name') {
$name = (string) $function['id'];
if (isset($this->importedSymbols[$name])) {
$symbol = $this->importedSymbols[$name];
$callable = 'python\\' . str_replace('.', '\\', $symbol['module']) . '\\' . $symbol['member'];
} elseif (isset($this->definedFunctions[$name])) {
$callable = $name;
} elseif ($this->isPythonBuiltin($name)) {
$callable = 'python\\' . $name;
} else {
$callable = $this->variable($name);
}
} elseif (($function['_type'] ?? '') === 'Attribute') {
$callable = $this->attribute($function);
} else {
$callable = '(' . $this->expression($function) . ')';
}
$arguments = [];
foreach ($node['args'] ?? [] as $argument) {
$arguments[] = $this->expression($argument);
}
foreach ($node['keywords'] ?? [] as $keyword) {
$arguments[] = ($keyword['arg'] === null ? '...' : $keyword['arg'] . ': ')
. $this->expression($keyword['value']);
}
return $callable . '(' . implode(', ', $arguments) . ')';
}
/** @param array<string, mixed> $node */
private function attribute(array $node): string
{
$parts = [];
$cursor = $node;
while (($cursor['_type'] ?? '') === 'Attribute') {
array_unshift($parts, (string) $cursor['attr']);
$cursor = $cursor['value'];
}
if (($cursor['_type'] ?? '') === 'Name' && isset($this->moduleAliases[$cursor['id']])) {
return $cursor['id'] . '\\' . implode('\\', $parts);
}
$result = $this->expression($cursor);
foreach ($parts as $part) {
$result .= '->' . $part;
}
return $result;
}
/** @param array<string, mixed> $node */
private function binaryExpression(array $node): string
{
$operator = $node['op']['_type'] ?? '';
if ($operator === 'FloorDiv') {
return 'python\\operator\\floordiv(' . $this->expression($node['left']) . ', '
. $this->expression($node['right']) . ')';
}
if ($operator === 'MatMult') {
return 'python\\operator\\matmul(' . $this->expression($node['left']) . ', '
. $this->expression($node['right']) . ')';
}
return $this->expression($node['left']) . ' ' . $this->binaryOperator($node['op'], $node)
. ' ' . $this->expression($node['right']);
}
/** @param array<string, mixed> $operator @param array<string, mixed> $owner */
private function binaryOperator(array $operator, array $owner): string
{
return match ($operator['_type'] ?? '') {
'Add' => '+', 'Sub' => '-', 'Mult' => '*', 'Div' => '/', 'Mod' => '%',
'Pow' => '**', 'LShift' => '<<', 'RShift' => '>>', 'BitOr' => '|',
'BitXor' => '^', 'BitAnd' => '&',
default => $this->unsupported($owner, 'unsupported binary operator ' . ($operator['_type'] ?? 'unknown')),
};
}
/** @param array<string, mixed> $node */
private function unaryExpression(array $node): string
{
$operator = match ($node['op']['_type'] ?? '') {
'USub' => '-', 'UAdd' => '+', 'Not' => '!', 'Invert' => '~',
default => $this->unsupported($node, 'unsupported unary operator'),
};
return $operator . $this->expression($node['operand']);
}
/** @param array<string, mixed> $node */
private function comparison(array $node): string
{
if (count($node['ops'] ?? []) !== 1 || count($node['comparators'] ?? []) !== 1) {
$this->unsupported($node, 'chained comparisons require explicit temporary variables');
}
$left = $this->expression($node['left']);
$right = $this->expression($node['comparators'][0]);
return match ($node['ops'][0]['_type'] ?? '') {
'Eq' => $left . ' == ' . $right,
'NotEq' => $left . ' != ' . $right,
'Is' => $left . ' === ' . $right,
'IsNot' => $left . ' !== ' . $right,
'Lt' => $left . ' < ' . $right,
'LtE' => $left . ' <= ' . $right,
'Gt' => $left . ' > ' . $right,
'GtE' => $left . ' >= ' . $right,
'In' => 'python\\operator\\contains(' . $right . ', ' . $left . ')',
'NotIn' => '!python\\operator\\contains(' . $right . ', ' . $left . ')',
default => $this->unsupported($node, 'unsupported comparison operator'),
};
}
/** @param array<string, mixed> $node */
private function target(array $node): string
{
if (($node['_type'] ?? '') === 'Attribute' && $this->attributeStartsWithModuleAlias($node)) {
$this->unsupported($node, 'Python module attributes cannot be assigned or deleted');
}
return match ($node['_type'] ?? '') {
'Name' => $this->variable((string) $node['id']),
'Attribute' => $this->attribute($node),
'Subscript' => $this->expression($node['value']) . '[' . $this->slice($node['slice']) . ']',
default => $this->unsupported($node, 'unsupported assignment target'),
};
}
private function nameExpression(string $name): string
{
if (isset($this->moduleAliases[$name])) {
throw new RuntimeException(
"{$this->filename}: a Python module cannot be used as a first-class value in TypePHP namespace syntax",
);
}
if (isset($this->importedSymbols[$name])) {
$symbol = $this->importedSymbols[$name];
return 'python\\' . str_replace('.', '\\', $symbol['module']) . '\\' . $symbol['member'];
}
return $this->variable($name);
}
private function variable(string $name): string
{
return '$' . ($name === 'this' ? 'this_' : $name);
}
private function constant(mixed $value): string
{
if (is_array($value) && isset($value['_python_constant'])) {
throw new RuntimeException("{$this->filename}: Python {$value['_python_constant']} literals are not supported yet");
}
if ($value === null) {
return 'null';
}
if (is_bool($value)) {
return $value ? 'true' : 'false';
}
return var_export($value, true);
}
/** @param list<array<string, mixed>> $nodes */
private function expressionList(array $nodes): string
{
$expressions = [];
foreach ($nodes as $node) {
$expressions[] = $this->expression($node);
}
return implode(', ', $expressions);
}
/** @param array<string, mixed> $node */
private function dictionaryItems(array $node): string
{
$items = [];
foreach ($node['keys'] ?? [] as $index => $key) {
if ($key === null) {
$items[] = '...' . $this->expression($node['values'][$index]);
} else {
$items[] = $this->expression($key) . ' => ' . $this->expression($node['values'][$index]);
}
}
return implode(', ', $items);
}
/** @param array<string, mixed> $node */
private function slice(array $node): string
{
if (($node['_type'] ?? '') !== 'Slice') {
return $this->expression($node);
}
return 'python\\slice('
. (($node['lower'] ?? null) === null ? 'null' : $this->expression($node['lower'])) . ', '
. (($node['upper'] ?? null) === null ? 'null' : $this->expression($node['upper'])) . ', '
. (($node['step'] ?? null) === null ? 'null' : $this->expression($node['step'])) . ')';
}
/** @param array<string, mixed> $node */
private function joinedString(array $node): string
{
$parts = [];
foreach ($node['values'] ?? [] as $value) {
if (($value['_type'] ?? '') === 'FormattedValue') {
if (($value['format_spec'] ?? null) !== null || ($value['conversion'] ?? -1) !== -1) {
$this->unsupported($value, 'formatted f-string conversions are not supported yet');
}
$parts[] = '(' . $this->expression($value['value']) . ')->toString()';
} else {
$parts[] = $this->expression($value);
}
}
return $parts === [] ? "''" : implode(' . ', $parts);
}
private function isPythonBuiltin(string $name): bool
{
static $builtins = [
'abs', 'all', 'any', 'bool', 'bytes', 'callable', 'dict', 'dir', 'enumerate',
'filter', 'float', 'getattr', 'hasattr', 'int', 'isinstance', 'issubclass', 'iter',
'len', 'list', 'map', 'max', 'min', 'next', 'object', 'open', 'ord', 'pow', 'print',
'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'str',
'sum', 'tuple', 'type', 'vars', 'zip',
];
return in_array($name, $builtins, true);
}
/** @param array<string, mixed> $function @return array<string, true> */
private function functionLocalNames(array $function): array
{
$locals = [];
$globals = [];
$arguments = $function['args'] ?? [];
foreach (array_merge($arguments['posonlyargs'] ?? [], $arguments['args'] ?? [], $arguments['kwonlyargs'] ?? []) as $argument) {
$locals[(string) $argument['arg']] = true;
}
foreach (['vararg', 'kwarg'] as $kind) {
if (($arguments[$kind] ?? null) !== null) {
$locals[(string) $arguments[$kind]['arg']] = true;
}
}
$stack = array_reverse($function['body'] ?? []);
while ($stack !== []) {
$value = array_pop($stack);
if (!is_array($value)) {
continue;
}
if (($value['_type'] ?? '') === 'FunctionDef') {
continue;
}
if (($value['_type'] ?? '') === 'Global') {
foreach ($value['names'] ?? [] as $name) {
$globals[(string) $name] = true;
}
continue;
}
if (($value['_type'] ?? '') === 'Name' && ($value['ctx']['_type'] ?? '') === 'Store') {
$locals[(string) $value['id']] = true;
}
foreach ($value as $item) {
if (is_array($item)) {
$stack[] = $item;
}
}
}
foreach ($globals as $name => $_) {
unset($locals[$name]);
}
return $locals;
}
/** @param list<string> $names @return list<string> */
private function variables(array $names): array
{
$variables = [];
foreach ($names as $name) {
$variables[] = $this->variable((string) $name);
}
return $variables;
}
/** @param array<string, mixed> $node */
private function attributeStartsWithModuleAlias(array $node): bool
{
$cursor = $node;
while (($cursor['_type'] ?? '') === 'Attribute') {
$cursor = $cursor['value'];
}
return ($cursor['_type'] ?? '') === 'Name' && isset($this->moduleAliases[$cursor['id']]);
}
private function line(string $code): string
{
return str_repeat(' ', $this->indent) . $code;
}
private function safeComment(string $value): string
{
return str_replace(['*/', "\r", "\n"], ['* /', ' ', ' '], $value);
}
/** @param array<string, mixed> $node */
private function unsupported(array $node, ?string $detail = null): never
{
$line = (int) ($node['lineno'] ?? 0);
$type = (string) ($node['_type'] ?? 'unknown');
$message = "{$this->filename}:{$line}: unsupported Python syntax {$type}";
if ($detail !== null) {
$message .= ": {$detail}";
}
throw new RuntimeException($message);
}
}

@ -0,0 +1,176 @@
<?php
namespace TypePhp\PythonTools\IdeHelper;
final class HelperRenderer
{
/** @var array<string, true> */
private const RESERVED = [
'__halt_compiler' => true, 'abstract' => true, 'and' => true, 'array' => true,
'as' => true, 'bool' => true, 'break' => true, 'callable' => true, 'case' => true,
'catch' => true, 'class' => true, 'clone' => true, 'const' => true, 'continue' => true,
'declare' => true, 'default' => true, 'die' => true, 'do' => true, 'echo' => true,
'else' => true, 'elseif' => true, 'empty' => true, 'enddeclare' => true,
'endfor' => true, 'endforeach' => true, 'endif' => true, 'endswitch' => true,
'endwhile' => true, 'enum' => true, 'eval' => true, 'exit' => true, 'extends' => true,
'false' => true, 'final' => true, 'finally' => true, 'float' => true, 'fn' => true,
'for' => true, 'foreach' => true, 'from' => true, 'function' => true, 'global' => true,
'goto' => true, 'if' => true, 'implements' => true, 'include' => true,
'include_once' => true, 'instanceof' => true, 'insteadof' => true, 'int' => true,
'interface' => true, 'isset' => true, 'iterable' => true, 'list' => true,
'match' => true, 'mixed' => true, 'namespace' => true, 'never' => true, 'new' => true,
'null' => true, 'object' => true, 'or' => true, 'parent' => true, 'print' => true,
'private' => true, 'protected' => true, 'public' => true, 'readonly' => true,
'require' => true, 'require_once' => true, 'resource' => true, 'return' => true,
'self' => true, 'static' => true, 'string' => true, 'switch' => true, 'throw' => true,
'trait' => true, 'true' => true, 'try' => true, 'unset' => true, 'use' => true,
'var' => true, 'void' => true, 'while' => true, 'xor' => true, 'yield' => true,
];
/**
* Render declarations inside an unreachable branch. IDEs can index them,
* while accidentally including the helper has no runtime side effects.
*
* @param array<string, mixed> $metadata
*/
public function render(array $metadata): string
{
$module = (string) $metadata['module'];
$namespace = $module === 'builtins'
? 'python'
: 'python\\' . str_replace('.', '\\', $module);
$lines = [
'<?php',
'',
'/**',
' * @generated TypePHP Python IDE helper.',
' * This file is for IDE indexing and must not be executed or compiled.',
' */',
'',
'namespace ' . $namespace . ';',
'',
];
$seenFunctions = [];
foreach ($metadata['attributes'] ?? [] as $attribute) {
$name = (string) ($attribute['name'] ?? '');
if (!$this->isDeclarableName($name)) {
$lines[] = '// Omitted Python attribute with an invalid PHP identifier: ' . $this->comment($name);
continue;
}
$lines[] = 'const ' . $name . ' = new \\PyObject();';
$lines[] = '';
}
foreach ($metadata['functions'] ?? [] as $function) {
$name = (string) ($function['name'] ?? '');
$folded = strtolower($name);
if (!$this->isDeclarableName($name) || isset($seenFunctions[$folded])) {
$lines[] = '// Omitted Python callable not representable as a PHP function: ' . $this->comment($name);
continue;
}
$seenFunctions[$folded] = true;
$lines[] = 'function ' . $name . '(' . $this->renderParameters($function['parameters'] ?? [])
. '): \\PyObject ' . $this->unreachableBody();
$lines[] = '';
}
$seenClasses = [];
foreach ($metadata['classes'] ?? [] as $class) {
$name = (string) ($class['name'] ?? '');
$folded = strtolower($name);
if (!$this->isDeclarableName($name) || isset($seenClasses[$folded])) {
$lines[] = '// Omitted Python class not representable as a PHP class: ' . $this->comment($name);
continue;
}
$seenClasses[$folded] = true;
if (!isset($seenFunctions[$folded])) {
$seenFunctions[$folded] = true;
$lines[] = 'function ' . $name . '(' . $this->renderParameters($class['parameters'] ?? [])
. '): ' . $name . ' ' . $this->unreachableBody();
$lines[] = '';
}
$properties = $class['properties'] ?? [];
if ($properties !== []) {
$lines[] = '/**';
foreach ($properties as $property) {
if ($this->isValidIdentifier((string) $property)) {
$lines[] = ' * @property \\PyObject $' . $property;
}
}
$lines[] = ' */';
}
$lines[] = 'class ' . $name . ' extends \\PyObject';
$lines[] = '{';
$lines[] = ' public function __construct(' . $this->renderParameters($class['parameters'] ?? [])
. ') { parent::__construct(); }';
$seenMethods = [];
foreach ($class['methods'] ?? [] as $method) {
$methodName = (string) ($method['name'] ?? '');
$methodFolded = strtolower($methodName);
if ($methodFolded === 'count') {
$lines[] = " // Python count() conflicts with PyObject::count(); use __call('count', [...]).";
continue;
}
if (!$this->isValidIdentifier($methodName) || isset($seenMethods[$methodFolded])) {
continue;
}
$seenMethods[$methodFolded] = true;
$lines[] = ' public function ' . $methodName . '('
. $this->renderParameters($method['parameters'] ?? []) . '): \\PyObject '
. $this->unreachableBody();
}
$lines[] = '}';
$lines[] = '';
}
$lines[] = 'die(\\PyObject::IDE_HELPER_ONLY);';
$lines[] = '';
return implode(PHP_EOL, $lines);
}
/** @param list<array{name: string, optional?: bool, variadic?: bool}> $parameters */
private function renderParameters(array $parameters): string
{
$regular = [];
$variadic = null;
$optionalSeen = false;
foreach ($parameters as $index => $parameter) {
$name = (string) ($parameter['name'] ?? ('arg' . $index));
if (!$this->isValidIdentifier($name) || $name === 'this') {
$name = 'arg' . $index;
}
if (!empty($parameter['variadic'])) {
$variadic ??= 'mixed ...$' . $name;
continue;
}
$optional = $optionalSeen || !empty($parameter['optional']);
$optionalSeen = $optional;
$regular[] = 'mixed $' . $name . ($optional ? ' = null' : '');
}
if ($variadic !== null) {
$regular[] = $variadic;
}
return implode(', ', $regular);
}
private function isDeclarableName(string $name): bool
{
return $this->isValidIdentifier($name) && !isset(self::RESERVED[strtolower($name)]);
}
private function isValidIdentifier(string $name): bool
{
return preg_match('/^[a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]*$/D', $name) === 1;
}
private function comment(string $value): string
{
return str_replace(["\r", "\n", '*/'], [' ', ' ', '* /'], $value);
}
private function unreachableBody(): string
{
return '{ die(\\PyObject::IDE_HELPER_ONLY); }';
}
}

@ -0,0 +1,120 @@
<?php
namespace TypePhp\PythonTools\IdeHelper;
use JsonException;
use RuntimeException;
use Throwable;
final class PhpyModuleScanner
{
private const INSPECTOR = <<<'PYTHON'
import importlib
import inspect
import json
def typephp_parameters(value):
try:
result = []
for parameter in inspect.signature(value).parameters.values():
kind = str(parameter.kind)
result.append({
'name': parameter.name,
'optional': parameter.default is not inspect._empty or kind == 'KEYWORD_ONLY',
'variadic': kind in ('VAR_POSITIONAL', 'VAR_KEYWORD'),
})
return result
except Exception:
return [{'name': 'args', 'optional': True, 'variadic': True}]
def typephp_class(name, value):
methods = []
properties = []
for member in dir(value):
if not member or member.startswith('_'):
continue
try:
item = getattr(value, member)
if inspect.isroutine(item):
parameters = typephp_parameters(item)
if parameters and parameters[0]['name'] in ('self', 'cls'):
parameters.pop(0)
methods.append({'name': member, 'parameters': parameters})
else:
properties.append(member)
except Exception:
pass
return {
'name': name,
'parameters': typephp_parameters(value),
'methods': methods,
'properties': properties,
}
module = importlib.import_module(module_name)
metadata = {
'module': module_name,
'doc': getattr(module, '__doc__', '') or '',
'attributes': [],
'functions': [],
'classes': [],
}
for name in dir(module):
if not name or name.startswith('_'):
continue
try:
value = getattr(module, name)
if inspect.isclass(value):
metadata['classes'].append(typephp_class(name, value))
elif inspect.isroutine(value):
metadata['functions'].append({
'name': name,
'parameters': typephp_parameters(value),
})
else:
metadata['attributes'].append({'name': name})
except Exception:
pass
metadata_json = json.dumps(metadata, ensure_ascii=False)
PYTHON;
/** @return array<string, mixed> */
public function scan(string $moduleName): array
{
if (!extension_loaded('phpy')) {
throw new RuntimeException('The phpy extension is required to generate a Python IDE helper');
}
if (preg_match('/^[A-Za-z_]\w*(?:\.[A-Za-z_]\w*)*$/D', $moduleName) !== 1) {
throw new RuntimeException("Invalid Python module name: {$moduleName}");
}
try {
// Perform reflection entirely in Python. Besides reducing boundary
// crossings, this avoids retaining PHPy's short-lived Zend method
// trampolines at AOT call sites.
$result = \PyCore::eval(self::INSPECTOR, ['module_name' => $moduleName]);
$json = \PyCore::scalar($result->metadata_json);
if (!is_string($json)) {
throw new RuntimeException('Python inspector returned a non-string result');
}
$metadata = json_decode($json, true, 512, JSON_THROW_ON_ERROR);
if (!is_array($metadata)) {
throw new RuntimeException('Python inspector returned invalid metadata');
}
return $metadata;
} catch (JsonException $exception) {
throw new RuntimeException(
"Unable to decode metadata for Python module `{$moduleName}`: {$exception->getMessage()}",
0,
$exception,
);
} catch (Throwable $exception) {
throw new RuntimeException(
"Unable to inspect Python module `{$moduleName}`: {$exception->getMessage()}",
0,
$exception,
);
}
}
}

@ -0,0 +1,65 @@
<?php
namespace TypePhp\PythonTools\IdeHelper;
final class PyObjectHelperRenderer
{
public function render(): string
{
$die = '{ die(self::IDE_HELPER_ONLY); }';
$lines = [
'<?php',
'',
'/**',
' * @generated TypePHP Python IDE helper.',
' * This file is for IDE indexing and must not be executed or compiled.',
' */',
'',
'class PyObject implements \\ArrayAccess, \\Iterator, \\Countable',
'{',
" public const IDE_HELPER_ONLY = 'IDE helper only';",
'',
' public function __construct(mixed $value = null) {}',
' public function __call(string $name, array $arguments): mixed ' . $die,
' public function __get(string $name): mixed ' . $die,
' public function __set(string $name, mixed $value): void {}',
' public function __unset(string $name): void {}',
' public function __toString(): string ' . $die,
' public function toArray(): array ' . $die,
' public function toValue(): mixed ' . $die,
'',
' /*',
' * TypePHP keyword methods are compiler intrinsics.',
' * They do not exist on the runtime PyObject class.',
' */',
' public function toInt(): int ' . $die,
' public function toFloat(): float ' . $die,
' public function toString(): string ' . $die,
' public function toBool(): bool ' . $die,
' public function toStream(): mixed ' . $die,
' public function toBigInt(): mixed ' . $die,
' public function toBigFloat(): mixed ' . $die,
' public function toDecimal(): mixed ' . $die,
' public function toObject(?string $class = null): object ' . $die,
' public function toAny(): mixed ' . $die,
' public function toRef(): mixed ' . $die,
'',
' public function __invoke(mixed ...$arguments): mixed ' . $die,
' public function offsetGet(mixed $offset): mixed ' . $die,
' public function offsetSet(mixed $offset, mixed $value): void {}',
' public function offsetUnset(mixed $offset): void {}',
' public function offsetExists(mixed $offset): bool ' . $die,
' public function key(): mixed ' . $die,
' public function next(): void {}',
' public function rewind(): void {}',
' public function valid(): bool ' . $die,
' public function current(): mixed ' . $die,
' public function count(): int ' . $die,
'}',
'die(PyObject::IDE_HELPER_ONLY);',
'',
];
return implode(PHP_EOL, $lines);
}
}

@ -272,6 +272,8 @@ class Translator extends Preprocessor
$climate->tab()->out('--march <arch> Target CPU instruction set (e.g. native, x86-64-v3, armv8-a)'); $climate->tab()->out('--march <arch> Target CPU instruction set (e.g. native, x86-64-v3, armv8-a)');
$climate->tab()->out('--target-platform <triple> Cross-compilation target triple (e.g. aarch64-linux-gnu)'); $climate->tab()->out('--target-platform <triple> Cross-compilation target triple (e.g. aarch64-linux-gnu)');
$climate->tab()->out('--wasm[=profile] Build WASI component (default) or browser output'); $climate->tab()->out('--wasm[=profile] Build WASI component (default) or browser output');
$climate->tab()->out('--gen-python-helper <module> [--output-dir <dir>] Generate a Python namespace IDE helper');
$climate->tab()->out('--convert-python-to-php <file.py> Convert Python source to TypePHP source');
$climate->tab()->out('--lto Enable Link Time Optimization (-flto)'); $climate->tab()->out('--lto Enable Link Time Optimization (-flto)');
$climate->tab()->out('--no-literal-strings Disable literal strings optimization'); $climate->tab()->out('--no-literal-strings Disable literal strings optimization');
$climate->tab()->out('--php-version <ver> PHP language version to accept (8.2-8.5, default: 8.5)'); $climate->tab()->out('--php-version <ver> PHP language version to accept (8.2-8.5, default: 8.5)');

@ -3,6 +3,7 @@ use TypePhp\Translator;
use TypePhp\Build\WasiToolchain; use TypePhp\Build\WasiToolchain;
use TypePhp\Build\WasiProjectConfig; use TypePhp\Build\WasiProjectConfig;
use TypePhp\Build\PhpxLocator; use TypePhp\Build\PhpxLocator;
use TypePhp\PythonTools\Command as PythonToolsCommand;
function main(int $argc, array $argv): void function main(int $argc, array $argv): void
{ {
@ -14,6 +15,14 @@ function main(int $argc, array $argv): void
define("ROOT_PATH", getcwd()); define("ROOT_PATH", getcwd());
} }
$pythonToolStatus = PythonToolsCommand::execute($argv);
if ($pythonToolStatus !== null) {
if ($pythonToolStatus !== 0) {
exit($pythonToolStatus);
}
return;
}
if (getenv('TYPEPHP_WASM_INTERNAL_COMPILE') !== '1' && shouldCompileWasm($argv)) { if (getenv('TYPEPHP_WASM_INTERNAL_COMPILE') !== '1' && shouldCompileWasm($argv)) {
compileWasmProgram($argv); compileWasmProgram($argv);
return; return;

@ -0,0 +1,41 @@
--TEST--
PyObject toArray() and toValue() explicitly return PHP values
--SKIPIF--
<?php
if (!extension_loaded('phpy')) {
die('skip phpy extension is not loaded');
}
?>
--FILE--
<?php
function main(): void
{
$list = python\list([1, 2, 3]);
$integer = python\int(42);
var_dump($list->toArray());
var_dump($list->toValue());
var_dump($integer->toValue());
var_dump($integer->toValue()->toInt());
}
?>
--EXPECT--
array(3) {
[0]=>
int(1)
[1]=>
int(2)
[2]=>
int(3)
}
array(3) {
[0]=>
int(1)
[1]=>
int(2)
[2]=>
int(3)
}
int(42)
int(42)

@ -1,53 +0,0 @@
--TEST--
PyObject toPlainValue() explicitly returns a PHP value
--SKIPIF--
<?php
if (!extension_loaded('phpy')) {
die('skip phpy extension is not loaded');
}
?>
--FILE--
<?php
function main(): void
{
$list = python\list([1, 2, 3]);
$integer = python\int(42);
var_dump(toPlainValue($list));
var_dump(toPlainValue($list)->toArray());
var_dump($integer->toPlainValue());
var_dump($integer->toPlainValue()->toInt());
try {
toPlainValue(new stdClass());
} catch (Error $error) {
var_dump(str_contains($error->getMessage(), 'supports PyObject only'));
}
}
function toPlainValue(mixed $value): mixed
{
return $value->toPlainValue();
}
?>
--EXPECT--
array(3) {
[0]=>
int(1)
[1]=>
int(2)
[2]=>
int(3)
}
array(3) {
[0]=>
int(1)
[1]=>
int(2)
[2]=>
int(3)
}
int(42)
int(42)
bool(true)
Loading…
Cancel
Save