- Move Chinese docs to docs/zh-cn/ (44 md + RUNTIME_LIFECYCLE.html) - Move existing English docs to docs/en/ - Translate all Chinese docs to English under docs/en/ - Update README.md links to docs/en/ and README-CN.md to docs/zh-cn/master
parent
7f0a8181f5
commit
3189d5bd7f
94 changed files with 17667 additions and 58 deletions
@ -0,0 +1,341 @@ |
||||
# AOT Compilation Speed Optimization Research Notes |
||||
|
||||
This document records the current assessment of Swoole-Compiler AOT compilation speed, bottleneck analysis, and future research directions. It is not yet tied to any specific PR. |
||||
|
||||
## Goal |
||||
|
||||
Reduce the following two categories of time cost: |
||||
|
||||
1. **Cold-start full build**: `./bin/tpc.php project.yml` |
||||
2. **Hot-start incremental build**: recompiling after changing only a few PHP files |
||||
|
||||
The focus is on large projects and compiler self-hosting scenarios. |
||||
|
||||
## Current Pipeline |
||||
|
||||
The main flow is in `src/Php/Translator.php`: |
||||
|
||||
1. `prepare()`: scan files, parse AST, collect symbols, sort dependencies |
||||
2. `convert()`: generate the corresponding `.cc` for each PHP file |
||||
3. `genStubFile()`: generate the arginfo / class register header file |
||||
4. `genFunctionDeclarations()` / `genDataDeclarations()`: generate the build-time internal declaration header |
||||
5. `genExtension()`: generate a single `extension-<target>.cc` |
||||
6. `compile()`: compile all `.cc/.c/...` into `.o` |
||||
7. `build()`: link into the final executable or extension |
||||
|
||||
## Main Bottleneck Assessment |
||||
|
||||
### 1. Common Header Churn Causing Full Recompilation |
||||
|
||||
Currently all translation units include: |
||||
|
||||
- `php_<target>_func_decl.h` |
||||
- `php_<target>_data_decl.h` |
||||
|
||||
As soon as any function declaration, default-argument helper, or global symbol declaration changes, a large number of `.cc` files get recompiled. |
||||
|
||||
This is one of the key reasons for the poor incremental build performance of large projects. |
||||
|
||||
### 2. The `extension-<target>.cc` Single File Is Too Large |
||||
|
||||
The extension main file carries: |
||||
|
||||
- class entry registration |
||||
- the function table |
||||
- literal strings |
||||
- module initialization |
||||
- static property initialization |
||||
- constant initialization |
||||
|
||||
The larger the project, the larger this single TU becomes, making it easy to become a compilation tail bottleneck; even if other files can be compiled in parallel, they all stall on this one large file. |
||||
|
||||
### 3. Missing a General Incremental Cache |
||||
|
||||
Currently only `phpx/src/misc` has an object cache: |
||||
|
||||
- `hasMiscObjectFileCache()` |
||||
|
||||
The `.cc`, common headers, arginfo headers, and extension file generated for the user's own project are still essentially fully regenerated and fully recompiled. |
||||
|
||||
### 4. clang-format Overhead Is Fixed and Serial |
||||
|
||||
`formatCppCode()` runs once for each generated file: |
||||
|
||||
```bash |
||||
clang-format -i <file> |
||||
``` |
||||
|
||||
This introduces: |
||||
|
||||
- extra process startup overhead |
||||
- a large amount of disk I/O |
||||
- serial formatting waits |
||||
|
||||
It is especially noticeable for large projects. |
||||
|
||||
### 5. arginfo / stub Are Regenerated Every Time |
||||
|
||||
`generateStubFile()` currently runs every time; even if the input PHP files have not changed, it regenerates the header files, further amplifying the header churn problem. |
||||
|
||||
### 6. Only the Compilation Stage Is Parallel; the Front Stages Are Mostly Serial |
||||
|
||||
Currently `compileWithPcntl()` only parallelizes `.cc -> .o`: |
||||
|
||||
- prepare |
||||
- convert |
||||
- stub generation |
||||
- format |
||||
|
||||
These stages are still mostly serial. |
||||
|
||||
## Highest-priority Optimization Directions |
||||
|
||||
## P0: Do Not Rewrite Files When Content Is Unchanged |
||||
|
||||
This is the most worthwhile foundational change to prioritize. |
||||
|
||||
### Idea |
||||
|
||||
For all generated files: |
||||
|
||||
- `.cc` |
||||
- arginfo `.h` |
||||
- `php_<target>_func_decl.h` |
||||
- `php_<target>_data_decl.h` |
||||
- `extension-<target>.cc` |
||||
|
||||
Compare the content before writing to disk: |
||||
|
||||
- Same content: **do not write the file** |
||||
- Different content: write it |
||||
|
||||
### Value |
||||
|
||||
Avoid triggering downstream full recompilation merely because of an mtime change. |
||||
|
||||
--- |
||||
|
||||
## P0: General object cache / incremental compilation |
||||
|
||||
Extend the current caching approach that only targets `phpx/src/misc` to user-generated code. |
||||
|
||||
### Suggested Cache Conditions |
||||
|
||||
For each target `.o`: |
||||
|
||||
1. `.o` exists |
||||
2. `.o` is newer than its corresponding source file |
||||
3. `.o` is newer than the headers it depends on |
||||
4. The compile option signature has not changed (optimization level, debug, sanitize, cxxflags, PHP/ZTS, etc.) |
||||
|
||||
When satisfied, skip compilation directly. |
||||
|
||||
### Supporting Requirements |
||||
|
||||
A clear "build signature" mechanism is needed, for example: |
||||
|
||||
- compiler backend |
||||
- cpp compiler path |
||||
- C++ standard |
||||
- optimize/debug/sanitize |
||||
- build mode |
||||
- PHP/ZTS information |
||||
|
||||
--- |
||||
|
||||
## P0: Disable clang-format by Default |
||||
|
||||
It is recommended to make formatting an explicit capability rather than part of the default compilation path. |
||||
|
||||
### Recommendation |
||||
|
||||
- Disable by default |
||||
- Add a `--format` or debug/dev mode to enable it |
||||
- Or format only changed files |
||||
|
||||
### Value |
||||
|
||||
This is a low-risk optimization with immediate effect. |
||||
|
||||
--- |
||||
|
||||
## P1: Split the Common Declaration Header |
||||
|
||||
### Current Problem |
||||
|
||||
Complex default-argument helpers also enter the common `func_decl.h`, widening the impact of header changes. |
||||
|
||||
### Optional Directions |
||||
|
||||
1. **Split declaration headers by source file** |
||||
2. **Move helpers from the common header to local headers / local `.cc`** |
||||
3. **Only truly cross-TU declarations go into the common header** |
||||
|
||||
### Goal |
||||
|
||||
Reduce "one change, full project recompilation". |
||||
|
||||
--- |
||||
|
||||
## P1: Split `extension-<target>.cc` |
||||
|
||||
### Splittable Modules |
||||
|
||||
1. `extension-main.cc` |
||||
2. `extension-class-register-*.cc` |
||||
3. `extension-function-table.cc` |
||||
4. `extension-const-init.cc` |
||||
5. `extension-static-init.cc` |
||||
|
||||
### Value |
||||
|
||||
- Reduce the size of a single TU |
||||
- Enhance parallel compilation benefits |
||||
- Reduce the tail wait for large projects |
||||
|
||||
--- |
||||
|
||||
## P1: arginfo / stub caching |
||||
|
||||
### Direction |
||||
|
||||
Introduce input-content-based caching for `generateStubFile()`: |
||||
|
||||
- Source PHP content hash |
||||
- gen_stub version signature |
||||
- PHP version signature |
||||
|
||||
Do not overwrite output header files when the content is unchanged. |
||||
|
||||
### Value |
||||
|
||||
Reduce header churn, with a clear effect when combined with incremental builds. |
||||
|
||||
--- |
||||
|
||||
## P2: Parallelizing prepare / convert / stub generation |
||||
|
||||
Currently only the compile stage is parallelized. Later the following can be explored: |
||||
|
||||
1. Layering by dependency topology after file scanning |
||||
2. Parallel convert for files in the same layer |
||||
3. Parallel stub generation for files in the same layer |
||||
|
||||
### Risk Points |
||||
|
||||
- There is a lot of shared state (literalStrings, classMap, funcMap, propMap, symbol tables, etc.) |
||||
- It is necessary to first sort out which state can be sharded and which must be merged |
||||
|
||||
Therefore this direction has large benefits but also higher implementation complexity. |
||||
|
||||
--- |
||||
|
||||
## P2: Symbol-dependency-driven minimal recompilation |
||||
|
||||
An ideal incremental build should not be based only on file timestamps, but on: |
||||
|
||||
- which symbols have changed |
||||
- which files depend on those symbols |
||||
|
||||
### Goal |
||||
|
||||
When modifying one PHP file, rebuild only: |
||||
|
||||
1. the file itself |
||||
2. files that depend on its exported symbols |
||||
3. the necessary extension / declaration modules |
||||
|
||||
This would significantly improve hot build speed for large projects. |
||||
|
||||
--- |
||||
|
||||
## P2: Toolchain-level optimization |
||||
|
||||
### Compilation caches |
||||
|
||||
- `ccache` |
||||
- `sccache` |
||||
|
||||
### Faster linkers |
||||
|
||||
- `mold` |
||||
- `lld` |
||||
|
||||
### Precompiled headers |
||||
|
||||
Try PCH for stable large headers, for example: |
||||
|
||||
- `phpx.h` |
||||
- `phpx_helper.h` |
||||
- `phpx_std.h` |
||||
|
||||
These optimizations are relatively cheap to implement and can be advanced together with the compiler option layer. |
||||
|
||||
## Special Constraints on Literal Arrays |
||||
|
||||
Literal arrays are different from literal strings: |
||||
|
||||
- **Literal strings** can leverage permanent strings to bypass the Zend request lifecycle |
||||
- **Literal arrays** must exist from `module_init()` to `module_clean()`, i.e. between PHP's `RINIT/RSHUTDOWN` |
||||
|
||||
Therefore all future "array initialization caching" research must obey: |
||||
|
||||
1. **PHP arrays must not be persisted into process-level permanent objects** |
||||
2. Only the "initialization plan" or "generated code template" can be cached |
||||
3. Real array objects must be constructed within the request lifecycle |
||||
|
||||
The already-introduced `ArrayInitPlan` belongs to this kind of safe abstraction: |
||||
|
||||
- It only saves `expr/init/clean` |
||||
- It does not save array object instances that cross requests |
||||
|
||||
## Suggested Landing Order |
||||
|
||||
### Phase One (Fastest Effect) |
||||
|
||||
1. Do not write files when content is unchanged |
||||
2. General `.o` cache |
||||
3. Disable clang-format by default |
||||
4. arginfo/stub content cache |
||||
|
||||
### Phase Two (Structural Benefits) |
||||
|
||||
5. Split the common header |
||||
6. Split `extension-<target>.cc` |
||||
7. Narrow the visibility of default-argument helpers |
||||
|
||||
### Phase Three (Long-term Optimization) |
||||
|
||||
8. prepare/convert parallelization |
||||
9. Symbol-dependency-driven minimal recompilation |
||||
10. PCH / ccache / mold / sccache |
||||
|
||||
## Suggested Code Locations to Research First |
||||
|
||||
- `src/Php/Translator.php` |
||||
- `formatCppCode()` |
||||
- `compile()` |
||||
- `compileSourceFile()` |
||||
- `compileWithPcntl()` |
||||
- `genFunctionDeclarations()` |
||||
- `genDataDeclarations()` |
||||
- `genExtension()` |
||||
- `genStubFile()` |
||||
- `src/Php/Backend/*` |
||||
- Compile/link command construction, convenient for integrating `ccache` / `mold` / `lld` |
||||
|
||||
## A Realistic Assessment |
||||
|
||||
For large projects, slow AOT compilation is usually not simply "g++ is slow", but the superposition of the following: |
||||
|
||||
1. Full regeneration |
||||
2. Header churn causing full recompilation |
||||
3. A single oversized extension TU |
||||
4. Per-file formatting |
||||
5. Lack of a real incremental cache |
||||
|
||||
Therefore the most effective direction is not to first tune compilation flags, but to prioritize: |
||||
|
||||
- **incrementality** |
||||
- **splitting** |
||||
- **reducing the common dependency surface** |
||||
@ -0,0 +1,662 @@ |
||||
# In-Place Optimization Plan for High-Precision Types |
||||
|
||||
## 1. Background |
||||
|
||||
TypePHP currently implements `BigInt`, `BigFloat`, and `Decimal` as PHPX `Box` objects stored in Zend resources. High-precision operations use an immutable result interface, for example: |
||||
|
||||
```cpp |
||||
target = php::BigInt::mul(target, rhs); |
||||
``` |
||||
|
||||
Even when the PHP source uses compound assignment: |
||||
|
||||
```php |
||||
$target *= $rhs; |
||||
``` |
||||
|
||||
The compiler still generates code that "creates a new result and reassigns". Taking BigInt as an example, a single multiplication currently typically requires: |
||||
|
||||
1. Create a new `BigInt` Box. |
||||
2. Register a new Zend resource. |
||||
3. Initialize a new `mpz_t`. |
||||
4. Allocate GMP limb storage for the computed result. |
||||
5. Move-assign the new resource to the target variable. |
||||
6. Destruct the old resource, Box, and underlying numeric storage. |
||||
|
||||
In scenarios such as loops, accumulation, factorials, and monetary aggregation, these overheads grow linearly with the number of operations: |
||||
|
||||
```php |
||||
for ($i = 0; $i < $count; $i++) { |
||||
$value = $value * 1000; |
||||
} |
||||
``` |
||||
|
||||
The underlying objects of GMP, MPFR, and mpdecimal are not immutable; all three support in-place operations where the output overlaps the input. The current immutable behavior comes from PHPX's high-precision API, not from a limitation of the underlying math libraries. |
||||
|
||||
This document provides the implementation plan, semantic constraints, phased plan, and acceptance criteria for in-place operations on high-precision types. |
||||
|
||||
## 2. Optimization Goals |
||||
|
||||
### 2.1 Primary Goals |
||||
|
||||
- Reuse existing objects for uniquely-held BigInt, BigFloat, and Decimal Boxes. |
||||
- Reuse GMP limb, MPFR mantissa, and mpdecimal coefficient storage as much as possible. |
||||
- Eliminate the result Box and Zend resource temporary objects in compound assignments. |
||||
- Use `php::Int` or `php::Var` directly for native RHS values such as integers, avoiding the construction of high-precision RHS Boxes. |
||||
- Fuse the safe `$x = $x {op} $rhs` pattern into an in-place operation. |
||||
- Preserve PHP's value semantics, reference semantics, evaluation order, and exception behavior. |
||||
- Automatically fall back to the current immutable implementation in unsafe or unprovably-safe scenarios. |
||||
|
||||
### 2.2 Non-Goals |
||||
|
||||
- The first phase does not optimize complex lvalues such as array elements, dynamic properties, and property hooks. |
||||
- It does not rely on whole-program alias analysis to guarantee correctness. |
||||
- It does not modify GMP, MPFR, or mpdecimal third-party source code. |
||||
- It does not turn all ordinary binary expressions into mutable computations. |
||||
- It does not change the implicit conversion rules between different high-precision types. |
||||
|
||||
## 3. Underlying Library Capabilities |
||||
|
||||
| Type | Underlying object | In-place operation | Memory reuse characteristics | |
||||
|---|---|---|---:| |
||||
| BigInt | GMP `mpz_t` / `mpz_class` | Supported | Reuses limb when capacity is sufficient, grows only when the result grows | |
||||
| BigFloat | MPFR `mpfr_t` | Supported | Currently fixed 256-bit precision, ordinary operations can usually keep reusing the mantissa | |
||||
| Decimal | mpdecimal `mpd_t` / `decimal::Decimal` | Supported | Can reuse the coefficient; the library itself provides `operator+=` and other in-place interfaces | |
||||
|
||||
Typical in-place calls are as follows: |
||||
|
||||
```cpp |
||||
mpz_mul(dst, dst, rhs); |
||||
mpfr_mul(dst, dst, rhs, MPFR_RNDN); |
||||
mpd_qmul(dst, dst, rhs, context, &status); |
||||
``` |
||||
|
||||
mpdecimal's C++ wrapper already provides: |
||||
|
||||
```cpp |
||||
Decimal::operator+= |
||||
Decimal::operator-= |
||||
Decimal::operator*= |
||||
Decimal::operator/= |
||||
Decimal::operator%= |
||||
``` |
||||
|
||||
Therefore the technical bottleneck lies mainly in PHP Box sharing semantics, exception safety, and the compiler's evaluation order, rather than in the math libraries themselves. |
||||
|
||||
## 4. Language Semantics That Must Be Preserved |
||||
|
||||
### 4.1 Copy-on-write |
||||
|
||||
A high-precision value may be shared by multiple PHP variables through the same Box: |
||||
|
||||
```php |
||||
$a = std::bigInt(10); |
||||
$b = $a; |
||||
$a *= 2; |
||||
``` |
||||
|
||||
The result must be: |
||||
|
||||
```text |
||||
$a = 20 |
||||
$b = 10 |
||||
``` |
||||
|
||||
The shared Box must not be modified directly. PHPX must check the Zend resource reference count before operating: |
||||
|
||||
- Resource uniquely held: modify the original Box directly. |
||||
- Resource shared: copy the Box, bind the target variable to the copy, then modify the copy. |
||||
|
||||
Runtime copy-on-write is the last line of defense for correctness. Compiler static analysis only reduces unnecessary checks and identifies fusable expressions; it cannot replace the runtime check. |
||||
|
||||
### 4.2 PHP References |
||||
|
||||
In the following scenario, two variables point to the same PHP reference container: |
||||
|
||||
```php |
||||
$a = std::bigInt(10); |
||||
$b =& $a; |
||||
$a *= 2; |
||||
``` |
||||
|
||||
The result must be that both `$a` and `$b` become 20. The in-place API must operate on the actual zval inside the reference through `Variant::unwrap_ptr()`; when copy-on-write occurs, it should update the value in the reference container rather than rebinding the PHPX wrapper object. |
||||
|
||||
### 4.3 RHS and Target Variable Aliasing |
||||
|
||||
The following must be handled correctly: |
||||
|
||||
```php |
||||
$a *= $a; |
||||
``` |
||||
|
||||
It is recommended that the in-place interface take the target by reference and the RHS by value: |
||||
|
||||
```cpp |
||||
BigInt::mulAssign(Variant &target, Variant rhs); |
||||
``` |
||||
|
||||
If the RHS shares the same resource as the target, the RHS's temporary reference count will cause copy-on-write to take the copy branch. This may miss an in-place opportunity, but it naturally guarantees correctness. A dedicated path for "RHS and target are the same Box" can be added later. |
||||
|
||||
### 4.4 Evaluation Order |
||||
|
||||
The following two pieces of code cannot be treated as equivalent in all cases: |
||||
|
||||
```php |
||||
$x = $x * changeValue($x); |
||||
$x *= changeValue($x); |
||||
``` |
||||
|
||||
The RHS may reassign, modify by reference, or modify `$x` through closure capture. C++ function argument evaluation order cannot be used to replace PHP's evaluation rules either. |
||||
|
||||
The compiler must follow these rules: |
||||
|
||||
- Use the existing ordered-operand and side-effect capture mechanism for true `AssignOp`. |
||||
- For `$x = $x {op} $rhs`, fuse only when the RHS does not write to or escape `$x`. |
||||
- Use the current "compute new result then assign" path when safety cannot be proven. |
||||
- If the old `$x` must be saved to preserve ordering, that temporary increases the reference count, and runtime copy-on-write fallback should be allowed automatically. |
||||
|
||||
### 4.5 Complex Lvalues |
||||
|
||||
The following expressions must not be rewritten in the first phase: |
||||
|
||||
```php |
||||
$array[getIndex()] = $array[getIndex()] * 2; |
||||
$object->value = $object->value * 2; |
||||
$object->hooked = $object->hooked * 2; |
||||
``` |
||||
|
||||
Reasons include: |
||||
|
||||
- The subscript expression may execute twice. |
||||
- The number of calls to getters, setters, or property hooks may change. |
||||
- Dynamic property reads/writes may trigger magic methods. |
||||
- The lvalue itself may have side effects. |
||||
|
||||
The first phase only supports simple local variables. Complex lvalues are designed separately in later phases through a "single-evaluation writable target" abstraction. |
||||
|
||||
### 4.6 Exception Safety |
||||
|
||||
The current immutable implementation computes the new result first and only assigns after success, so the target variable remains unchanged when an exception occurs: |
||||
|
||||
```php |
||||
$value = std::decimal('10'); |
||||
|
||||
try { |
||||
$value /= 0; |
||||
} catch (DivisionByZeroError $e) { |
||||
} |
||||
|
||||
echo $value; // still 10 |
||||
``` |
||||
|
||||
The in-place implementation must preserve this behavior. |
||||
|
||||
- BigInt: Check error conditions such as the divisor, modulus, and exponent before modifying. |
||||
- BigFloat: Check division-by-zero and error conditions explicitly defined by the current API before modifying. |
||||
- Decimal: `context.raise(status)` may throw after the underlying result has already been written, requiring a transactional commit or rollback mechanism. |
||||
- Memory allocation failure must also not leave the target in a partially-modified state. |
||||
|
||||
### 4.7 Resource identity |
||||
|
||||
High-precision Boxes are currently exposed as resources, and `get_resource_id()` and strict comparison may observe resource identity. In-place operations keep the resource id for uniquely-held variables, whereas the current immutable implementation generates a new resource id. |
||||
|
||||
One of the following contracts must be clarified before implementation: |
||||
|
||||
1. High-precision types are value types; resource identity is an internal implementation detail and is not guaranteed to remain unchanged across operations. |
||||
2. The current resource identity change must be preserved, in which case only the underlying numeric storage can be reused and the resource must be rewrapped, reducing the benefit. |
||||
|
||||
Option 1 is recommended, and the high-precision type documentation should make it explicit: users should compare values and should not rely on internal resource ids. The value semantics of shared variables are still strictly guaranteed by copy-on-write. |
||||
|
||||
## 5. PHPX Design |
||||
|
||||
### 5.1 Explicit In-Place API |
||||
|
||||
It is not recommended to add high-precision operator overloading to the generic `Variant`. Explicit interfaces should be added to each high-precision type: |
||||
|
||||
```cpp |
||||
class BigInt { |
||||
public: |
||||
static Variant &addAssign(Variant &target, Variant rhs); |
||||
static Variant &subAssign(Variant &target, Variant rhs); |
||||
static Variant &mulAssign(Variant &target, Variant rhs); |
||||
static Variant &divAssign(Variant &target, Variant rhs); |
||||
static Variant &modAssign(Variant &target, Variant rhs); |
||||
}; |
||||
``` |
||||
|
||||
BigFloat and Decimal use the same naming convention. BigInt should also cover bitwise operations and shifts: |
||||
|
||||
```cpp |
||||
bitAndAssign |
||||
bitOrAssign |
||||
bitXorAssign |
||||
bitShiftLeftAssign |
||||
bitShiftRightAssign |
||||
``` |
||||
|
||||
The interface returns `Variant &`, so that compound assignment can still be used as an expression: |
||||
|
||||
```php |
||||
$result = ($value *= 2); |
||||
``` |
||||
|
||||
If the actual generated code is inconvenient to handle the reference return, a statement-only `void` fast path can be provided at the same time, but the assignment expression semantics must not be sacrificed. |
||||
|
||||
### 5.2 Box Uniqueness Utility |
||||
|
||||
Provide a reusable C++17 helper inside PHPX instead of duplicating Zend resource logic across the three types: |
||||
|
||||
```cpp |
||||
template <typename T> |
||||
T *separateBoxForWrite(Variant &target); |
||||
``` |
||||
|
||||
Responsibilities include: |
||||
|
||||
1. Dereference indirect/reference zvals. |
||||
2. Verify that target is the target Box type. |
||||
3. Check the Zend resource reference count. |
||||
4. Return the original Box when uniquely held. |
||||
5. Copy the Box when shared, and update the target through `Variant` assignment semantics. |
||||
6. Preserve typed reference checks and exception propagation. |
||||
|
||||
All three Boxes must support correct copying: |
||||
|
||||
- BigInt: copy the `mpz_class`. |
||||
- BigFloat: initialize at the source precision and copy the `mpfr_t`. |
||||
- Decimal: copy the `decimal::Decimal`. |
||||
|
||||
### 5.3 RHS Extraction |
||||
|
||||
The in-place interface should accept `Variant rhs` directly and reuse the existing operand extractor: |
||||
|
||||
- `php::Int` is converted directly to an underlying integer operand. |
||||
- `php::Var` checks its actual type at runtime. |
||||
- When already a Box of the same type, read the underlying value directly. |
||||
- Strings, floats, and different high-precision types continue to follow the current conversion restrictions. |
||||
|
||||
The generated code should prioritize: |
||||
|
||||
```cpp |
||||
php::BigInt::mulAssign(value, 1000L); |
||||
php::Decimal::mulAssign(value, factor); |
||||
``` |
||||
|
||||
Avoid: |
||||
|
||||
```cpp |
||||
php::BigInt::mulAssign(value, php::toBigInt(1000L)); |
||||
php::Decimal::mulAssign(value, php::toDecimal(1000L)); |
||||
``` |
||||
|
||||
For Decimal's integer RHS, mpdecimal's `_i64`/`_u64` interfaces can be used further to avoid constructing a temporary `decimal::Decimal`: |
||||
|
||||
```cpp |
||||
mpd_qmul_i64(result, left, rhs, context, &status); |
||||
``` |
||||
|
||||
### 5.4 BigInt Implementation Strategy |
||||
|
||||
BigInt prioritizes true in-place operations: |
||||
|
||||
```cpp |
||||
Variant &BigInt::mulAssign(Variant &target, Variant rhs) { |
||||
BigIntOperand right; |
||||
// Extract and validate the RHS first. |
||||
// Then perform copy-on-write on target. |
||||
// Finally call mpz_mul(dst, dst, right). |
||||
return target; |
||||
} |
||||
``` |
||||
|
||||
All recoverable error checks, such as division by zero, modulo by zero, and illegal shift amounts, must be completed before modifying. GMP capacity growth is managed internally; the original limb storage is reused when capacity is sufficient. |
||||
|
||||
### 5.5 BigFloat Implementation Strategy |
||||
|
||||
BigFloat currently uniformly uses `BIG_FLOAT_DEFAULT_PRECISION`, which is suitable for direct in-place operations: |
||||
|
||||
```cpp |
||||
mpfr_mul(dst, dst, rhs, MPFR_RNDN); |
||||
``` |
||||
|
||||
If per-object precision is supported in the future, the relationship between the non-in-place result precision and the compound-assignment target precision must be specified, and tests for objects of different precisions must be added. |
||||
|
||||
### 5.6 Decimal Implementation Strategy |
||||
|
||||
Decimal is implemented in two steps. |
||||
|
||||
The first step uses exception-safe transactional commit: |
||||
|
||||
```cpp |
||||
decimal::Decimal temporary; |
||||
uint32_t status = 0; |
||||
mpd_qmul(temporary.get(), current.getconst(), rhs, context, &status); |
||||
context.raise(status); |
||||
current = std::move(temporary); |
||||
``` |
||||
|
||||
This approach can eliminate the result Box and Zend resource, but still creates an underlying Decimal temporary object. |
||||
|
||||
The second step evaluates true in-place operations: |
||||
|
||||
- Complete explicit checks such as division-by-zero before modifying. |
||||
- Identify which status/trap values may throw after the operation. |
||||
- Provide backup/rollback for operations that may throw, or only perform in-place when it can be proven that no trap will be triggered. |
||||
- Run dedicated tests for Overflow, InvalidOperation, DivisionByZero, and simulated allocation failure. |
||||
|
||||
"Target value partially modified after an exception" must not be accepted for the sake of performance. |
||||
|
||||
## 6. Compiler Design |
||||
|
||||
### 6.1 True Compound Assignment |
||||
|
||||
First modify the existing Big* `AssignOp` generation path: |
||||
|
||||
```php |
||||
$value *= $rhs; |
||||
``` |
||||
|
||||
From: |
||||
|
||||
```cpp |
||||
value = php::BigInt::mul(value, rhs); |
||||
``` |
||||
|
||||
To: |
||||
|
||||
```cpp |
||||
php::BigInt::mulAssign(value, rhs); |
||||
``` |
||||
|
||||
Support matrix: |
||||
|
||||
| Type | First-phase operators | |
||||
|---|---| |
||||
| BigInt | `+= -= *= /= %= &= |= ^= <<= >>=` | |
||||
| BigFloat | `+= -= *= /=` | |
||||
| Decimal | `+= -= *= /= %=` | |
||||
|
||||
### 6.2 Ordinary Assignment Fusion |
||||
|
||||
Identify the following AST: |
||||
|
||||
```php |
||||
$x = $x {op} $rhs; |
||||
``` |
||||
|
||||
Fuse only when all of the following conditions are met: |
||||
|
||||
- The lvalue is a simple named variable. |
||||
- The left operand of the binary expression is the same variable. |
||||
- The variable's static type is BigInt, BigFloat, or Decimal. |
||||
- The operator is in the corresponding type's supported list. |
||||
- The RHS does not contain an assignment to, a reference acquisition of, or a known by-reference argument passing of the target variable. |
||||
- The RHS does not contain `eval`, dynamic calls, or other escape paths that cannot be safely analyzed; or the existing side-effect analysis clearly proves safety. |
||||
- The current expression context can correctly receive the in-place interface's return value. |
||||
|
||||
The following scenarios are not fused in the first phase: |
||||
|
||||
```php |
||||
$x = 2 - $x; |
||||
$x = $x * ($x = 2); |
||||
$x = $x * dynamicCall(); |
||||
$array[$key] = $array[$key] * 2; |
||||
$object->value = $object->value * 2; |
||||
``` |
||||
|
||||
Optimization of commutative operations such as `$x = $rhs + $x` or `$x = $rhs * $x` is deferred to later phases to avoid expanding the scope of the first version. |
||||
|
||||
### 6.3 Failure Fallback |
||||
|
||||
The optimization must be an optional codegen path: |
||||
|
||||
```text |
||||
Can safely operate in-place -> emit *Assign() |
||||
Cannot prove safety -> emit the current new-result path |
||||
``` |
||||
|
||||
Any type uncertainty, complex lvalue, reference escape, or side-effect analysis failure must not cause a compilation error; it should only lose that optimization. |
||||
|
||||
### 6.4 Relationship with SSA/Optimizer |
||||
|
||||
The initial version can perform local AST matching in `AssignOpTrait` and ordinary assignment resolution without relying on a complete SSA. |
||||
|
||||
Later, SSA can provide: |
||||
|
||||
- Whether the target variable has aliases. |
||||
- Whether the RHS writes to the target variable. |
||||
- Whether the variable escapes to dynamic calls or references. |
||||
- Whether it can statically prove the Box is uniquely held. |
||||
|
||||
Even if SSA proves uniqueness, the PHPX runtime copy-on-write check is still recommended to be retained, unless there is a strict escape proof and dedicated tests. |
||||
|
||||
## 7. Phased Implementation Plan |
||||
|
||||
### Phase 0: Baseline and Observation |
||||
|
||||
- Add test helper facilities for counting high-precision Box/resource creation. |
||||
- Establish benchmarks for BigInt, BigFloat, and Decimal loop operations. |
||||
- Record current wall time, Box count, resource count, and underlying allocation count. |
||||
- Freeze the current aliasing, reference, exception, and resource identity behavior. |
||||
|
||||
Deliverable: a baseline report and behavior tests, with no change to generated code. |
||||
|
||||
### Phase 1: Native RHS Fast Path |
||||
|
||||
- BigInt operations directly accept `php::Int`. |
||||
- BigFloat operations directly accept `php::Int`, `php::Float`. |
||||
- Decimal operations directly accept `php::Int` and `php::Var` that is actually an int. |
||||
- The Decimal integer path prioritizes `mpd_q*_i64`. |
||||
- Eliminate the high-precision Box the compiler creates for the RHS. |
||||
|
||||
Deliverable: no more unnecessary `toBigInt()`, `toBigFloat()`, `toDecimal()` on the RHS. |
||||
|
||||
### Phase 2: PHPX Copy-on-write Infrastructure |
||||
|
||||
- Implement `separateBoxForWrite<T>()`. |
||||
- Complete copy tests for the three Box types. |
||||
- Cover ordinary variables, shared variables, PHP references, indirect zvals, and RHS being the same Box. |
||||
- Clarify the resource identity contract. |
||||
|
||||
Deliverable: standalone PHPX unit tests, with no modification to the compiler generation path. |
||||
|
||||
### Phase 3: BigInt and BigFloat Compound Assignment |
||||
|
||||
- Implement the BigInt `*Assign()` method family. |
||||
- Implement the BigFloat `*Assign()` method family. |
||||
- Modify the generated code for true PHP `AssignOp`. |
||||
- Preserve fallback for unsafe paths. |
||||
- Run the full PHPX test suite, full compiler PHPUnit, relevant PHPT, and bootstrap compilation. |
||||
|
||||
Deliverable: syntax such as `$x *= $rhs` uses true in-place operations. |
||||
|
||||
### Phase 4: Ordinary Assignment Fusion |
||||
|
||||
- Identify simple local variables `$x = $x {op} $rhs`. |
||||
- Implement target variable write/escape checks. |
||||
- Prioritize enabling for pure-literal and pure-variable RHS. |
||||
- Preserve the old path for RHS with side effects. |
||||
|
||||
Deliverable: common patterns in the problem description no longer require users to manually convert to compound assignment. |
||||
|
||||
### Phase 5: Decimal Transactional In-Place Interface |
||||
|
||||
- Implement the Decimal `*Assign()` API. |
||||
- First use "underlying temporary result + commit on success". |
||||
- Optimize integer RHS using the `_i64` fast path. |
||||
- Cover all Decimal traps and the target value after exceptions. |
||||
|
||||
Deliverable: eliminate the Decimal result Box/resource while maintaining strong exception safety. |
||||
|
||||
### Phase 6: Decimal True In-Place Computation |
||||
|
||||
- Analyze the status/trap values each operator may trigger. |
||||
- Directly use the target `mpd_t` for operations that can be proven safe. |
||||
- Preserve the transactional path for high-risk operations. |
||||
- Determine through benchmarks whether the complexity is worthwhile. |
||||
|
||||
Deliverable: common Decimal accumulation operations reuse coefficient storage. |
||||
|
||||
### Phase 7: Complex Lvalues and Further Optimizations |
||||
|
||||
- Design a single-evaluation writable target abstraction. |
||||
- Evaluate support for array elements, static properties, and ordinary properties. |
||||
- Property hooks, magic methods, and dynamic properties are not enabled by default unless the number of calls and ordering can be strictly preserved. |
||||
- Evaluate commutative expression fusion and SSA uniqueness proof. |
||||
|
||||
## 8. Test Plan |
||||
|
||||
### 8.1 PHPX Unit Tests |
||||
|
||||
Each type and each operator must at least cover: |
||||
|
||||
- Unique Box in-place update. |
||||
- Shared Box triggers copy-on-write. |
||||
- PHP references update the same referenced value. |
||||
- RHS and target are the same Box. |
||||
- Allowed RHS types such as Int, Float, String, and Var. |
||||
- Exceptions for illegal RHS types. |
||||
- Edge cases such as division by zero, modulo by zero, and negative exponents. |
||||
- The target value remains unchanged after an exception. |
||||
- Capacity growth triggered by extremely large numbers. |
||||
- Multiple consecutive operations. |
||||
|
||||
### 8.2 Compiler PHPUnit |
||||
|
||||
Check the generated code: |
||||
|
||||
- `AssignOp` generates calls such as `BigInt::mulAssign()`. |
||||
- `$x = $x * 1000` is fused. |
||||
- RHS native integers no longer construct Big* Boxes. |
||||
- No fusion when the RHS has side effects. |
||||
- Array elements and properties are not fused in the first phase. |
||||
- Unsupported operators continue to produce the original FatalError. |
||||
|
||||
### 8.3 PHPT |
||||
|
||||
At least cover: |
||||
|
||||
```php |
||||
$a *= 2; |
||||
$a = $a * 2; |
||||
$b = $a; $a *= 2; |
||||
$b =& $a; $a *= 2; |
||||
$a *= $a; |
||||
$a *= ($factor = 2); |
||||
$result = ($a *= 2); |
||||
``` |
||||
|
||||
And cover for the three high-precision types: |
||||
|
||||
- Positive, negative, and zero values. |
||||
- Extreme values and precision boundaries. |
||||
- All supported compound assignment operators. |
||||
- The lvalue after an exception. |
||||
- Consecutive updates in a loop. |
||||
|
||||
### 8.4 Integration Verification |
||||
|
||||
Each phase must at least execute: |
||||
|
||||
```bash |
||||
./vendor/bin/phpunit |
||||
php run-tests.php tests/compiler/bigint tests/compiler/bignumber tests/compiler/decimal |
||||
php bin/tpc.php project.yml |
||||
``` |
||||
|
||||
PHPX modifications must also run the full PHPX unit test suite. |
||||
|
||||
## 9. Performance Acceptance |
||||
|
||||
Performance tests must at least include: |
||||
|
||||
- Sizes of 1, 4, 16, 64, 256, and 1024 limb/decimal digits. |
||||
- RHS being small integers, same-type high-precision values, and dynamic `php::Var`. |
||||
- Unique Box and shared Box. |
||||
- Loops of 1 thousand, 100 thousand, and 1 million iterations. |
||||
- BigInt growth multiplication versus stable-capacity addition. |
||||
- BigFloat fixed-precision accumulation. |
||||
- Decimal fixed 50-digit precision accumulation. |
||||
|
||||
Functional acceptance criteria: |
||||
|
||||
- Compound assignment of a unique BigInt/BigFloat does not create a result Box/resource per iteration. |
||||
- Native RHS does not create a high-precision Box. |
||||
- Shared Box correctly triggers copy-on-write. |
||||
- All exception paths keep the target value unchanged. |
||||
- Bootstrap compilation and full test suites pass. |
||||
|
||||
Performance acceptance is based on baseline data and does not preset unrealistic fixed multiples. At least the following should be reported separately: |
||||
|
||||
- Total elapsed time. |
||||
- Box/resource creation counts. |
||||
- Underlying memory allocation counts and bytes. |
||||
- Peak memory. |
||||
- Copy-on-write hit rate and fallback rate. |
||||
|
||||
If an optimization path cannot reduce allocations, or causes clear regression in common non-in-place expressions, the old path should be retained or that sub-optimization should be reverted. |
||||
|
||||
## 10. Risks and Rollback Strategy |
||||
|
||||
Main risks: |
||||
|
||||
- Incorrect Box sharing determination causing other variables to be modified unexpectedly. |
||||
- References or indirect zvals being rebound instead of updated. |
||||
- RHS side effects changing the evaluation order. |
||||
- Decimal target value being polluted after an exception. |
||||
- Undocumented changes in resource identity behavior. |
||||
- In-place capacity growth failure leaving an invalid underlying object. |
||||
|
||||
Control measures: |
||||
|
||||
- All optimizations are concentrated in a standalone PHPX API and a single compiler codegen branch. |
||||
- Fall back to the old implementation when safety cannot be proven. |
||||
- Enable incrementally by type and by operator. |
||||
- Commit each phase independently, avoiding modifying too many semantics at once. |
||||
- Do not remove the existing immutable API until exception, aliasing, and reference tests are complete. |
||||
|
||||
Rollback only requires the compiler to regenerate: |
||||
|
||||
```cpp |
||||
target = Type::operation(target, rhs); |
||||
``` |
||||
|
||||
The original immutable API must be retained throughout the entire migration period. |
||||
|
||||
## 11. Recommended Priority |
||||
|
||||
Considering benefit, complexity, and risk, the recommended order is: |
||||
|
||||
1. BigFloat in-place compound assignment. |
||||
2. BigInt in-place compound assignment. |
||||
3. BigInt/BigFloat ordinary assignment fusion. |
||||
4. Decimal native integer RHS fast path. |
||||
5. Decimal transactional `*Assign()`. |
||||
6. Decimal true in-place computation. |
||||
7. Complex lvalues and SSA enhancements. |
||||
|
||||
BigFloat has fixed precision and is the easiest to stably reuse underlying memory; BigInt has broader applications and its overall benefit may be the largest; Decimal has the most complex exception and trap semantics, and its true in-place modification should be pushed last. |
||||
|
||||
## 12. Final Target Code |
||||
|
||||
For safe simple variables: |
||||
|
||||
```php |
||||
$value = $value * 1000; |
||||
``` |
||||
|
||||
The final generation: |
||||
|
||||
```cpp |
||||
php::BigInt::mulAssign(value, 1000L); |
||||
``` |
||||
|
||||
Runtime: |
||||
|
||||
```text |
||||
Unique Box: reuse Box, resource, and underlying storage in place |
||||
Shared Box: copy-on-write, then modify the new Box |
||||
Unsafe scenario: fall back to the current immutable result implementation |
||||
``` |
||||
|
||||
This design confines the performance optimization within verifiable boundaries while preserving the consistency of TypePHP with PHP assignment, reference, and exception semantics. |
||||
@ -0,0 +1,142 @@ |
||||
# TypePHP Compiler Command Line |
||||
|
||||
## Bash Autocompletion |
||||
|
||||
TypePHP provides Bash completion that is kept in sync with the current compiler arguments. To enable it temporarily in the current terminal: |
||||
|
||||
```shell |
||||
source <(./tpc --generate-completion=bash) |
||||
``` |
||||
|
||||
When developing from the source repository, you can also run `source completions/tpc.bash` directly. |
||||
|
||||
To install it for the current user and have it auto-loaded in subsequent Bash sessions: |
||||
|
||||
```shell |
||||
mkdir -p "$HOME/.local/share/bash-completion/completions" |
||||
./tpc --generate-completion=bash \ |
||||
> "$HOME/.local/share/bash-completion/completions/tpc" |
||||
``` |
||||
|
||||
If your system does not automatically scan the user completion directory, you can load it in `~/.bashrc`: |
||||
|
||||
```shell |
||||
source "$HOME/.local/share/bash-completion/completions/tpc" |
||||
``` |
||||
|
||||
For a system-wide installation, write the generated output to `/usr/share/bash-completion/completions/tpc`. This operation typically |
||||
requires root privileges. |
||||
|
||||
The completion supports build options, WASM profiles, build modes, PHP/C++ versions, sanitizers, input sources, |
||||
project YAML, Python source files, and directory arguments. Everything after `--` is treated as arguments of the compiled program itself, and the completer |
||||
does not interpret them as `tpc` arguments anymore. |
||||
|
||||
Release packages ship a pre-generated `completions/tpc.bash`. This file is produced by the same generator, with unit |
||||
tests ensuring it matches the output of `./tpc --generate-completion=bash`. |
||||
|
||||
This document is kept in sync with `src/Translator.php::showUsage()`. Usage: |
||||
|
||||
```bash |
||||
bin/tpc.php <file|dir|project.yml> [options] [-- program-args...] |
||||
``` |
||||
|
||||
## Common Examples |
||||
|
||||
```bash |
||||
# Compile a single file |
||||
bin/tpc.php app.php |
||||
|
||||
# Optimize and run; arguments after `--` are passed to the generated program |
||||
bin/tpc.php app.php -O2 -r -- --flag value |
||||
|
||||
# Compile a project configuration |
||||
bin/tpc.php project.yml -O2 -j 8 |
||||
|
||||
# Generate a PHP extension |
||||
bin/tpc.php extension/ -m ext -o my_extension |
||||
|
||||
# Only generate C++, without compiling and linking |
||||
bin/tpc.php app.php --dry --build-dir /tmp/typephp-build |
||||
``` |
||||
|
||||
## Build Options |
||||
|
||||
| Option | Description | |
||||
|---|---| |
||||
| `-O <0-3>` | Optimization level, default `0`. | |
||||
| `-d`, `--debug` | Debug build; disables optimization and adds debug symbols and TypePHP source tracking. | |
||||
| `-o`, `--output <file>` | Output file name. | |
||||
| `-m`, `--mode <bin|lib|ext>` | Build mode, default `bin`. | |
||||
| `-r`, `--run` | Run after a successful build. | |
||||
| `-j`, `--job <num>` | Number of parallel compilation jobs, default `4`. | |
||||
| `-f`, `--force` | Ignore the phpx misc object cache and force recompilation. | |
||||
| `--build-dir <dir>` | Directory for generated C++ and intermediate artifacts. | |
||||
| `--dry` | Only generate C++, skipping compilation and linking. | |
||||
| `--format` | Run clang-format on the generated code. | |
||||
| `--no-progress` | Do not show the progress bar; output progress per file. | |
||||
| `--no-color` | Disable colored output. | |
||||
|
||||
`-v` / `--version` only displays the version; it is not a verbose option. |
||||
|
||||
## Target and Toolchain |
||||
|
||||
| Option | Description | |
||||
|---|---| |
||||
| `--php-version <8.4|8.5>` | Restrict the accepted PHP syntax version, default `8.5`. | |
||||
| `--cxx-std <ver>` | C++ standard, e.g. `c++17`, `c++20`. | |
||||
| `--march <arch>` | Target instruction set, e.g. `native`, `x86-64-v3`. | |
||||
| `--target-platform <triple>` | Cross-compilation target triple. | |
||||
| `--lto` | Enable Link Time Optimization. | |
||||
| `--sanitize <type>` | Enable a sanitizer, e.g. `address`, `undefined`. | |
||||
| `--no-console` | Windows GUI mode hides the console window. | |
||||
| `--profile` | Enable the gperftools profiler on Linux and force recompilation of related objects. | |
||||
|
||||
`--php-version` controls the source syntax accepted by the parser and is also used in `project.yml` to select source files based on `PHP_VERSION` / `PHP_VERSION_ID`. It is not responsible for choosing the PHP installation directory to link against. |
||||
|
||||
The minimum runtime version for both TypePHP and PHPX is PHP 8.4. `--php-version` and the actually linked `libphp.so` do not need to match exactly in minor version, but both must be PHP 8.4 or higher. |
||||
|
||||
## C++ Compilation and Link Arguments |
||||
|
||||
These arguments can all be repeated: |
||||
|
||||
```bash |
||||
-I /opt/library/include |
||||
-D FEATURE_ENABLED=1 |
||||
-L /opt/library/lib |
||||
-l curl |
||||
``` |
||||
|
||||
Corresponding long options: |
||||
|
||||
- `--include-path` |
||||
- `--define` |
||||
- `--link-path` |
||||
- `--link-lib` |
||||
|
||||
## Project Configuration Precedence |
||||
|
||||
When a `project.yml` is passed, command-line arguments take precedence over same-named settings in the YAML. For the project file format, see the user documentation and the project configuration parser in the code. |
||||
|
||||
### PHP Extension Dependencies |
||||
|
||||
When a program depends on other PHP extensions, the required modules can be written into the Zend module dependency table: |
||||
|
||||
```yaml |
||||
extension-dependencies: |
||||
- pdo_mysql |
||||
- curl |
||||
``` |
||||
|
||||
`ext-deps` is an equivalent shorthand name. Only one of these names can be used in a project; using both `extension-dependencies` and `ext-deps` produces a configuration error. |
||||
|
||||
The compiler generates a `ZEND_MOD_REQUIRED` for each entry. Zend checks whether these extensions are loaded when loading the TypePHP module. This setting does not represent native link libraries; C/C++ link dependencies still use `link-libs`. |
||||
|
||||
## Viewing the Authoritative Help |
||||
|
||||
The command-line implementation may continue to evolve; for released versions the actual arguments are determined by the following command: |
||||
|
||||
```bash |
||||
bin/tpc.php --help |
||||
``` |
||||
|
||||
For compatibility boundaries, see [INCOMPATIBLE_PHP_FEATURES.md](INCOMPATIBLE_PHP_FEATURES.md); for build modes, see [COMPILATION_MODES.md](COMPILATION_MODES.md). |
||||
@ -0,0 +1,96 @@ |
||||
# AOT compile-time functions and keyword methods |
||||
|
||||
This document records the compile-time functions, keyword methods, and related construction entry points that are specific to the AOT compiler. They are not part of standard PHP syntax, and an ordinary PHP runtime can only rely on the compatibility stubs provided by `src/polyfills.php`. |
||||
|
||||
## Core compile-time functions |
||||
|
||||
There are currently 5 core global compile-time functions. |
||||
|
||||
| Name | Parameters | Purpose | Current primary handling location | |
||||
| --- | --- | --- | --- | |
||||
| `any($value)` | 1 | Degrades the expression to `mixed/any`, preventing further processing as a static native/object type. | General function-call expression entry. | |
||||
| `refval($target)` | 1 | Explicitly passes a variable, array element, or object property by reference to a dynamic call or a call whose reference parameter cannot be statically identified. | Argument parsing, dynamic calls, SSA/optimizer reference escape analysis. | |
||||
| `objval($value, ClassName::class or 'ClassName')` | 2 | Tells the compiler that `$value` is an object of the specified class, and generates the `php::toObject(..., target_ce)` runtime fallback check. | Function-call resolution, object type inference. | |
||||
| `expected($condition)` | 1 | Marks the condition as usually true, generating the Zend `EXPECTED(...)` branch prediction macro. | General function-call expression entry. | |
||||
| `unexpected($condition)` | 1 | Marks the condition as usually false, generating the Zend `UNEXPECTED(...)` branch prediction macro. | General function-call expression entry. | |
||||
|
||||
Constraints: |
||||
|
||||
- `refval()` only accepts variables, array elements, or object properties. |
||||
- The second parameter of `objval()` must be a compile-time-resolvable class-name string or `ClassName::class`. |
||||
- `any()` can be used in any expression position; it directly expands its single argument at compile time without generating a runtime function call. |
||||
- `expected()` / `unexpected()` accept exactly one non-expanded argument and return bool; they are usually used in `if`, `elseif`, and loop conditions, and do not change the argument's evaluation count or true/false semantics. |
||||
|
||||
## Keyword methods |
||||
|
||||
There are currently 12 built-in keyword methods. |
||||
|
||||
| Name | Equivalent behavior | Description | |
||||
| --- | --- | --- | |
||||
| `toAny()` | `any($receiver)` | Returns the receiver itself, but with the type degraded to `mixed/any`. | |
||||
| `toRef()` | `refval($receiver)` | Returns a reference to the receiver; parameter restrictions are the same as `refval()`. | |
||||
| `toObject()` | `php::toObject($receiver)` | May take a target-class parameter, performing object conversion/checking. | |
||||
| `toInt()` | `php::toInt($receiver)` | Converts to a native int expression. | |
||||
| `toFloat()` | `php::toFloat($receiver)` | Converts to a native float expression. | |
||||
| `toString()` | `php::toString($receiver)` | Converts to a string expression. | |
||||
| `toBool()` | `php::toBool($receiver)` | Converts to a bool expression. | |
||||
| `toArray()` | `php::toArray($receiver)` | Converts to an array expression. | |
||||
| `toStream()` | `php::toStream($receiver)` | Converts to a stream expression. | |
||||
| `toBigInt()` | `php::BigInt::newInstance($receiver)` | Constructs a BigInt. | |
||||
| `toBigFloat()` | `php::BigFloat::newInstance($receiver)` | Constructs a BigFloat. | |
||||
| `toDecimal()` | `php::Decimal::newInstance($receiver)` | Constructs a Decimal. | |
||||
|
||||
Constraints: |
||||
|
||||
- `toAny()` and `toRef()` accept no parameters. |
||||
- `toRef()` only applies to receivers that can take references. |
||||
- Keyword methods take precedence over ordinary methods and universal method dispatch. |
||||
|
||||
## `std::` compile-time construction entry points |
||||
|
||||
There are currently 10 `std::` compile-time construction entry points. |
||||
|
||||
| Name | Purpose | Main limitation | |
||||
| --- | --- | --- | |
||||
| `std::int($value)` | Explicitly creates a native int expression. | Requires 1 value parameter. | |
||||
| `std::float($value)` | Explicitly creates a native float expression. | Requires 1 value parameter. | |
||||
| `std::bool($value)` | Explicitly creates a native bool expression. | Requires 1 value parameter. | |
||||
| `std::bigInt($value)` | Constructs a BigInt. | Implicit construction from a float variable is not allowed. | |
||||
| `std::decimal($value)` | Constructs a Decimal. | A float variable must be converted via string or integer; float literals are handled per the original literal. | |
||||
| `std::bigFloat($value)` | Constructs a BigFloat. | Requires 1 value parameter. | |
||||
| `std::array($type, $size[, ...$sizes])` | Constructs a fixed-size std array. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
| `std::vector($type[, $size])` | Constructs a std vector. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
| `std::map($keyType, $valueType)` | Constructs a std map. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
| `std::ordered_map($keyType, $valueType)` | Constructs a std ordered map. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
|
||||
## Std container conversion keyword methods |
||||
|
||||
There are currently 4 Std container conversion keyword methods. |
||||
|
||||
| Name | Purpose | Main limitation | |
||||
| --- | --- | --- | |
||||
| `toStdArray(...)` | Wraps the variable as a std array. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
| `toStdVector(...)` | Wraps the variable as a std vector. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
| `toStdMap(...)` | Wraps the variable as a std map. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
| `toStdOrderedMap(...)` | Wraps the variable as a std ordered map. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
|
||||
## Mechanisms not counted in this list |
||||
|
||||
- `$array->any()` is a universal method that maps to PHP `array_any()`, not the `any()` compile-time function. |
||||
- `Type::*` are compile-time type-description constants, not functions. |
||||
- keyword extension methods are a user-defined extension method mechanism and are not part of the fixed built-in compile-time function list. |
||||
|
||||
## Implementation constraints |
||||
|
||||
Compile-time functions should be usable in any legal expression position and maintain consistent semantics across all paths: |
||||
|
||||
- `any()` is already handled uniformly at the ordinary function-call expression entry; assignments, parameters, return values, array elements, and operator subexpressions share the same semantics. |
||||
- `refval()` / `toRef()` have many special cases in argument parsing and dynamic call paths and should later be unified into a single "reference-wrapping expression" resolution entry. |
||||
- `objval()` is currently recognized through the function-call resolution and type-inference paths and is relatively centralized. |
||||
- `expected()` / `unexpected()` generate `EXPECTED(...)` / `UNEXPECTED(...)` respectively at the ordinary function-call entry and produce no PHP runtime function call. |
||||
|
||||
Future refactoring goals: |
||||
|
||||
- Establish a unified `CompileTimeFunctionResolver` or equivalent module. |
||||
- Reuse the same compile-time function metadata in `parseExpr()` / `detectTypeOfExpr()` / `detectClassOfExpr()` / argument parsing paths. |
||||
- Continue unifying the behavior of `refval()` and `objval()` across different expression paths. |
||||
@ -0,0 +1,474 @@ |
||||
# TypePHP Core Class OOA / OOD / OOP Refactoring Plan |
||||
|
||||
## 1. Document Purpose |
||||
|
||||
This document guides the subsequent architectural refactoring of `Translator`, `CompilerBase`, and `Preprocessor`. Implementation should proceed phase by phase, without a one-shot rewrite. |
||||
|
||||
Current baseline: |
||||
|
||||
| Class | Lines | Methods | Current Role | |
||||
|---|---:|---:|---| |
||||
| `Translator` | 3717 | 126 | CLI, project configuration, code generation, build coordination | |
||||
| `CompilerBase` | 3843 | 208 | Compilation state, AST dispatch, Resolver, Emitter | |
||||
| `Preprocessor` | 973 | 25 | Declaration collection, AST lowering, dependency and semantic validation | |
||||
|
||||
Current inheritance structure: |
||||
|
||||
```text |
||||
Translator |
||||
extends Preprocessor |
||||
extends CompilerBase |
||||
``` |
||||
|
||||
Main problems: |
||||
|
||||
- The three classes form an inheritance-based God Object, where high-level flows can access all low-level mutable state. |
||||
- A large number of Traits only achieve physical splitting, and still implicitly depend on all `$this` fields of the host. |
||||
- Frontend analysis, name resolution, semantic validation, code generation, and native build lack clear boundaries. |
||||
- Arrays, AST attributes, and `string|false` are used as implicit protocols between modules. |
||||
- Manual state switches such as `resetFile()`, `resetClass()`, `resetFunction()` are easy to forget to restore. |
||||
|
||||
## 2. Refactoring Principles |
||||
|
||||
1. Behavior preservation takes priority; separate architecture refactoring from semantic changes into separate commits. |
||||
2. Establish object boundaries first, then remove old entry points; during migration, old and new implementations may coexist. |
||||
3. Prefer composition, interfaces, and immutable value objects; business Traits only as a transitional measure. |
||||
4. Handler Registry indexes directly by node class name, avoiding linear responsibility chains that degrade compilation performance. |
||||
5. Compilation state must be passed explicitly through Context or Session. |
||||
6. Resolver is responsible for decisions; Generator/Emitter is responsible for code generation; the two must not be mixed. |
||||
7. Each phase must have independent PHPUnit and corresponding PHPT regression evidence. |
||||
|
||||
## 3. OOA: Domain Object Analysis |
||||
|
||||
### 3.1 Compilation Session Domain |
||||
|
||||
Responsible for the state of one compilation lifecycle: |
||||
|
||||
```text |
||||
CompilationSession |
||||
CompilerConfiguration |
||||
ScopeStack |
||||
ScopeFrame |
||||
FileContext |
||||
ClassContext |
||||
FunctionContext |
||||
``` |
||||
|
||||
### 3.2 Frontend Analysis Domain |
||||
|
||||
Responsible for PHP source code to validated AST/model: |
||||
|
||||
```text |
||||
SourceParser |
||||
FrontendPipeline |
||||
DeclarationCollector |
||||
DependencyAnalyzer |
||||
SemanticAnalyzer |
||||
AstLoweringPass |
||||
``` |
||||
|
||||
### 3.3 Resolution Domain |
||||
|
||||
Responsible for symbol and language semantic decisions: |
||||
|
||||
```text |
||||
NameResolver |
||||
TypeResolver |
||||
MethodCallResolver |
||||
PropertyResolver |
||||
ConstantResolver |
||||
AccessPolicy |
||||
SymbolRepository |
||||
InheritanceGraph |
||||
``` |
||||
|
||||
### 3.4 Code Generation Domain |
||||
|
||||
Responsible for AST/entity model to C++: |
||||
|
||||
```text |
||||
ExpressionCompiler |
||||
StatementCompiler |
||||
ClassCodeGenerator |
||||
FunctionCodeGenerator |
||||
WrapperGenerator |
||||
ExtensionModuleGenerator |
||||
``` |
||||
|
||||
### 3.5 Build Domain |
||||
|
||||
Responsible for generated files to final artifacts: |
||||
|
||||
```text |
||||
SourcePipeline |
||||
NativeBuilder |
||||
ResourceCompiler |
||||
BuildModeStrategy |
||||
CompileOptions |
||||
LinkOptions |
||||
BuildResult |
||||
``` |
||||
|
||||
### 3.6 Application Entry Domain |
||||
|
||||
Responsible for user input and top-level flow: |
||||
|
||||
```text |
||||
CompilerApplication |
||||
CompileCommand |
||||
CompilerFacade |
||||
ProjectYamlLoader |
||||
CommandLineInput |
||||
``` |
||||
|
||||
## 4. OOD: Target Architecture |
||||
|
||||
```text |
||||
CompilerApplication |
||||
└─ CompilerFacade |
||||
├─ ProjectLoader |
||||
├─ SourcePipeline |
||||
├─ FrontendPipeline |
||||
│ ├─ DeclarationCollector |
||||
│ ├─ AstLoweringPass[] |
||||
│ ├─ DependencyAnalyzer |
||||
│ └─ SemanticAnalyzer |
||||
├─ TranslationCodeGenerator |
||||
│ ├─ ExpressionCompiler |
||||
│ ├─ StatementCompiler |
||||
│ ├─ ClassCodeGenerator |
||||
│ └─ FunctionCodeGenerator |
||||
└─ NativeBuilder |
||||
``` |
||||
|
||||
### 4.1 Translator's Target |
||||
|
||||
`Translator` ultimately acts as a Facade/Coordinator, only organizing the flow: |
||||
|
||||
```php |
||||
final class Translator |
||||
{ |
||||
public function translate(ProjectInput $input): BuildResult; |
||||
} |
||||
``` |
||||
|
||||
It is forbidden to continue assuming: |
||||
|
||||
- CLI argument parsing; |
||||
- AST node semantic determination; |
||||
- C++ template concatenation for classes, functions, and wrappers; |
||||
- shell command execution; |
||||
- preprocessor internal state. |
||||
|
||||
### 4.2 Preprocessor's Target |
||||
|
||||
`Preprocessor` becomes an independent Frontend Service, no longer extending `CompilerBase`: |
||||
|
||||
```php |
||||
final class Preprocessor |
||||
{ |
||||
public function process(SourceUnit $source, CompilationSession $session): PreprocessResult; |
||||
} |
||||
``` |
||||
|
||||
Organize Passes using the Pipeline pattern: |
||||
|
||||
```text |
||||
ParseSourcePass |
||||
→ NameResolutionPass |
||||
→ PropertyHookLoweringPass |
||||
→ DeclarationCollectionPass |
||||
→ TraitExpansionPass |
||||
→ InheritanceValidationPass |
||||
→ TypeValidationPass |
||||
→ DependencyCollectionPass |
||||
``` |
||||
|
||||
### 4.3 CompilerBase's Target |
||||
|
||||
`CompilerBase` is eventually replaced by the following objects: |
||||
|
||||
- `CompilationSession`: compilation lifecycle state; |
||||
- `ExpressionCompiler`: expression Handler dispatch; |
||||
- `StatementCompiler`: statement Handler dispatch; |
||||
- `CompilerServices`: the Resolver and Generator collections; |
||||
- `CodeGenerationContext`: generation-phase context. |
||||
|
||||
After the migration is complete, delete `CompilerBase`, or keep only a short-term compatibility Facade. |
||||
|
||||
## 5. Design Pattern Application |
||||
|
||||
### Facade |
||||
|
||||
`CompilerFacade` and the final `Translator` provide a stable top-level entry point, hiding the details of Frontend, Generator, and Builder. |
||||
|
||||
### Pipeline |
||||
|
||||
`FrontendPipeline` explicitly maintains the order of frontend Passes, and each Pass can be tested independently. |
||||
|
||||
### Handler Registry |
||||
|
||||
Expressions and statements are dispatched in O(1) by AST class name: |
||||
|
||||
```php |
||||
$handlers[Expr\MethodCall::class] = $methodCallHandler; |
||||
``` |
||||
|
||||
### Strategy |
||||
|
||||
Build modes are implemented by the following strategies: |
||||
|
||||
- `BinaryBuildStrategy` |
||||
- `ExtensionBuildStrategy` |
||||
- `LibraryBuildStrategy` |
||||
- `EmbedBuildStrategy` |
||||
|
||||
### Chain of Responsibility |
||||
|
||||
Method resolution order: |
||||
|
||||
```text |
||||
DeclaredMethodResolver |
||||
→ ObjectExtensionMethodResolver |
||||
→ UniversalMethodResolver |
||||
→ MagicCallResolver |
||||
→ DynamicCallResolver |
||||
``` |
||||
|
||||
Property resolution order: |
||||
|
||||
```text |
||||
BackingSlotResolver |
||||
→ PropertyHookResolver |
||||
→ DeclaredPropertyResolver |
||||
→ NativePropertyResolver |
||||
→ DynamicPropertyResolver |
||||
``` |
||||
|
||||
### Repository |
||||
|
||||
`SymbolRepository` uniformly manages functions, classes, interfaces, constants, and inheritance relationships; callers no longer handle Repository keys themselves. |
||||
|
||||
### State / Scope Stack |
||||
|
||||
Use `ScopeStack` and `ScopeGuard` to replace the reset series of methods, ensuring state restoration on exceptions, `Skip`, and `Redo`. |
||||
|
||||
### Value Object / Result Object |
||||
|
||||
Gradually introduce: |
||||
|
||||
- `SourceUnit` |
||||
- `SourceLocation` |
||||
- `GeneratedExpression` |
||||
- `GeneratedStatement` |
||||
- `ResolvedCall` |
||||
- `ResolvedPropertyAccess` |
||||
- `PreprocessResult` |
||||
- `TranslationResult` |
||||
- `BuildResult` |
||||
|
||||
## 6. OOP Incremental Implementation Phases |
||||
|
||||
### Phase 0: Architecture Protection Tests |
||||
|
||||
Tasks: |
||||
|
||||
- Establish an expression and statement node coverage checklist; |
||||
- Add frontend Pass order tests; |
||||
- Add Scope exception recovery tests; |
||||
- Add SymbolRepository name normalization tests; |
||||
- Fix the Property Hook, extension methods, inheritance, and exception test sets; |
||||
- Establish snapshots or structural assertions for key generated C++. |
||||
|
||||
Acceptance: |
||||
|
||||
- PHPUnit passes in full; |
||||
- Core PHPT all pass; |
||||
- Subsequent phases can identify Handler omissions and evaluation order changes. |
||||
|
||||
### Phase 1: CompilationSession and ScopeStack |
||||
|
||||
Tasks: |
||||
|
||||
1. Create `CompilationSession`, `CompilerConfiguration`, `ScopeStack`. |
||||
2. Move in current file, namespace, class, method, function, PHP version, and phase state. |
||||
3. `CompilerBase`'s old properties first proxy to the Session. |
||||
4. Replace the reset series of methods with `enter/leave` and `try/finally`. |
||||
5. Delete the proxy properties. |
||||
|
||||
Acceptance: |
||||
|
||||
- `CompilerBase` no longer directly owns scope state; |
||||
- Exceptions, `Skip`, `Redo` do not pollute the next scope; |
||||
- ScopeStack has independent unit tests. |
||||
|
||||
### Phase 2: Preprocessor Pipeline |
||||
|
||||
Tasks: |
||||
|
||||
1. Create `FrontendPass` and `FrontendPipeline`. |
||||
2. First migrate Property Hook lowering. |
||||
3. Migrate declaration collection and namespace/use handling. |
||||
4. Migrate dependency collection and file ordering. |
||||
5. Migrate Trait, inheritance, override, and interface implementation validation. |
||||
6. Remove `Preprocessor extends CompilerBase`. |
||||
|
||||
Acceptance: |
||||
|
||||
- Each Pass has independent tests; |
||||
- Pass order is defined in only one place; |
||||
- `Preprocessor.php` is kept within 200–300 lines. |
||||
|
||||
### Phase 3: ExpressionCompiler |
||||
|
||||
Tasks: |
||||
|
||||
1. Create `ExpressionHandlerRegistry` and `GeneratedExpression`. |
||||
2. Migrate scalar/const/variable, unary/binary/cast, array/assign in order. |
||||
3. Migrate function/method/static calls. |
||||
4. Migrate property, nullsafe, isset/empty/ref. |
||||
5. Migrate closure, generator, fiber, new, clone, instanceof. |
||||
6. Delete the old large `parseExpr()` dispatch. |
||||
|
||||
Acceptance: |
||||
|
||||
- Every supported Expr has a unique Handler; |
||||
- Handlers do not depend on `CompilerBase`; |
||||
- Registry checks for duplicates and omissions at startup; |
||||
- Evaluation order and side effect tests all pass. |
||||
|
||||
### Phase 4: StatementCompiler |
||||
|
||||
Tasks: |
||||
|
||||
1. Create `StatementHandlerRegistry` and `GeneratedStatement`. |
||||
2. Migrate return/echo, conditionals, loops, exception control flow. |
||||
3. Migrate global/static/namespace/declare. |
||||
4. Eliminate the shared `beforeStmtLines`, `afterStmtLines` protocol. |
||||
|
||||
Acceptance: |
||||
|
||||
- Statement Handlers return an explicit Result; |
||||
- Control flow generation is removed from `CompilerBase`; |
||||
- before/after statements are composed through Result. |
||||
|
||||
### Phase 5: Resolver Chain |
||||
|
||||
Tasks: |
||||
|
||||
1. Establish `MethodCallResolverChain`. |
||||
2. Establish `PropertyResolverChain`. |
||||
3. Establish `ConstantResolverChain`. |
||||
4. Establish a unified `AccessPolicy`. |
||||
5. Migrate `MethodCallTrait`, `PropertyAccessTrait`, `UniversalMethodCall`, `MagicMethodDetector` into the Resolver. |
||||
|
||||
Acceptance: |
||||
|
||||
- The priority of normal methods, extension methods, and `__call()` is defined in only one place; |
||||
- The priority of backing slot, Property Hook, and normal properties is defined in only one place; |
||||
- `private(set)`, `protected(set)` are determined only by AccessPolicy; |
||||
- Resolver no longer returns `string|false`. |
||||
|
||||
### Phase 6: Independent Code Generators |
||||
|
||||
Tasks: |
||||
|
||||
- Establish `ClassCodeGenerator`; |
||||
- Establish `FunctionCodeGenerator`; |
||||
- Establish `WrapperGenerator`; |
||||
- Establish `ExtensionModuleGenerator`; |
||||
- Move `parseClass()`, `parseFunction()`, wrapper, and registration code out of `Translator`. |
||||
|
||||
Acceptance: |
||||
|
||||
- Generator input is Entity/IR, output is `GeneratedFile`; |
||||
- `Translator` no longer directly concatenates concrete C++ templates; |
||||
- Key generation results have snapshot tests. |
||||
|
||||
### Phase 7: Translator Facade |
||||
|
||||
Tasks: |
||||
|
||||
1. Move CLI to `CompilerApplication` / `CompileCommand`. |
||||
2. `Translator` only injects ProjectLoader, Frontend, CodeGenerator, NativeBuilder. |
||||
3. Remove `Translator extends Preprocessor`. |
||||
4. Converge the public entry point to `translate(ProjectInput): BuildResult`. |
||||
|
||||
Acceptance: |
||||
|
||||
- `Translator` does not parse CLI; |
||||
- does not directly access the AST; |
||||
- does not directly execute shell; |
||||
- does not depend on Preprocessor internal state; |
||||
- the file is kept within 300–500 lines. |
||||
|
||||
### Phase 8: Remove the CompilerBase Inheritance Hierarchy |
||||
|
||||
Tasks: |
||||
|
||||
1. Clear remaining compatibility proxies and business Traits. |
||||
2. Move public query interfaces into explicit Services/Contexts. |
||||
3. Delete the `Translator → Preprocessor → CompilerBase` inheritance chain. |
||||
4. Delete uncalled legacy methods and fields. |
||||
|
||||
Acceptance: |
||||
|
||||
- Core components collaborate only through interfaces and DTOs; |
||||
- No business Trait implicitly accesses all host state; |
||||
- No core class exceeds about 800 lines; |
||||
- PHPUnit, core PHPT, and multi-PHP-version builds all pass. |
||||
|
||||
## 7. Per-Phase Execution Template |
||||
|
||||
Each phase is implemented following these steps: |
||||
|
||||
1. List the methods, fields, and call sites to migrate. |
||||
2. Add protection tests first. |
||||
3. Create the new interfaces, DTOs, and implementation. |
||||
4. Change old entry points to delegate to the new implementation. |
||||
5. Migrate call sites in batches. |
||||
6. Delete old implementations and proxy fields. |
||||
7. Run syntax checks, PHPUnit, and corresponding PHPT. |
||||
8. Check `git diff --check` and untracked build artifacts. |
||||
9. Update the phase status and actual deviations in this document. |
||||
|
||||
## 8. Recommended Directory Layout |
||||
|
||||
```text |
||||
src/ |
||||
├─ Application/ |
||||
├─ Compiler/ |
||||
│ └─ Scope/ |
||||
├─ Frontend/ |
||||
│ ├─ Pass/ |
||||
│ └─ Result/ |
||||
├─ CodeGeneration/ |
||||
│ ├─ Expression/ |
||||
│ └─ Statement/ |
||||
├─ Resolver/ |
||||
│ ├─ Call/ |
||||
│ ├─ Property/ |
||||
│ ├─ Constant/ |
||||
│ └─ Access/ |
||||
├─ Symbol/ |
||||
├─ Build/ |
||||
│ ├─ BuildMode/ |
||||
│ └─ Options/ |
||||
└─ Diagnostics/ |
||||
``` |
||||
|
||||
## 9. Phase Status |
||||
|
||||
| Phase | Status | |
||||
|---|---| |
||||
| 0. Architecture protection tests | Not started | |
||||
| 1. CompilationSession / ScopeStack | Not started | |
||||
| 2. Preprocessor Pipeline | Not started | |
||||
| 3. ExpressionCompiler | Not started | |
||||
| 4. StatementCompiler | Not started | |
||||
| 5. Resolver Chain | Not started | |
||||
| 6. Independent code generators | Not started | |
||||
| 7. Translator Facade | Not started | |
||||
| 8. Remove CompilerBase inheritance hierarchy | Not started | |
||||
|
||||
This table should be continuously updated during implementation; a phase must not be declared complete based solely on file line count. |
||||
@ -0,0 +1,356 @@ |
||||
# C++ Namespace, Prefix, and Symbol ABI Rules |
||||
|
||||
This document is the internal C++ naming convention for TypePHP, PHPX, and TypePHP-generated code. It solves the following problems: |
||||
|
||||
- Distinguishing TypePHP runtime logic, PHPX ZendAPI wrappers, project-private implementations, and user PHP symbols; |
||||
- Preventing framework helpers from generating the same C++ symbols as user-defined PHP functions or class methods; |
||||
- Clarifying which names are part of the stable ABI and which names are only for internal use within a single generated project; |
||||
- Providing a unified naming decision for adding helpers, caches, entry functions, and generated symbols. |
||||
|
||||
## 1. Overall Rules |
||||
|
||||
| Naming Domain | Meaning | Typical Form | Visibility Scope | ABI Property | |
||||
| --- | --- | --- | --- | --- | |
||||
| `typephp_` | TypePHP-specific runtime or compiled-artifact support logic | `typephp_call_parent_constructor()` | TypePHP/PHPX runtime | Internal or explicitly exported ABI | |
||||
| `php::` | C++ wrappers for PHP runtime capabilities such as ZendAPI, zval, HashTable, and call frames | `php::deindirect()` | PHPX C++ API | PHPX API | |
||||
| `typephp_<project>` | The private C++ namespace of a single compiled project | `namespace typephp_tpc` | Current generated project | Non-public ABI | |
||||
| `php_` | C++ callable symbols mapped from user PHP functions and class methods | `php_app__user__save()` | Visible to the linker | TypePHP/stub callable ABI | |
||||
|
||||
Core constraints: |
||||
|
||||
1. Do not add new global framework `php_*` helpers. |
||||
2. Capabilities that are unrelated to TypePHP and only wrap ZendAPI must be placed in `namespace php`. |
||||
3. Logic that is unique to TypePHP and needs to be called across generated files uses the `typephp_` prefix. |
||||
4. Data and functions that serve only one compiled project go into the `typephp_<project>` namespace. |
||||
5. Global `php_*` callable names are reserved for the compiled ABI of user PHP declarations. |
||||
|
||||
## 2. `typephp_`: TypePHP-specific Logic |
||||
|
||||
`typephp_` indicates that the API's semantics are defined by TypePHP and are not a general-purpose C++ wrapper of ZendAPI. Common scenarios include: |
||||
|
||||
- TypePHP property read/write rules; |
||||
- TypePHP construction, cloning, and parent method call chains; |
||||
- Runtime support for TypePHP compile-time Attributes; |
||||
- TypePHP-specific runtime logic such as Native Class and Property Hook; |
||||
- Initialization and shutdown entry points of the TypePHP embed runtime. |
||||
|
||||
Examples: |
||||
|
||||
```cpp |
||||
typephp_call_parent_constructor(object, constructor, args); |
||||
typephp_call_parent_clone(object, clone_method); |
||||
typephp_install_property_handlers(class_entry, handlers); |
||||
typephp_write_property_scoped(object, member, value, scope); |
||||
TYPEPHP_RUNTIME_INIT(project)(argc, argv); |
||||
``` |
||||
|
||||
### 2.1 Usage Boundaries |
||||
|
||||
- This prefix is the TypePHP internal C/C++ name space and does not represent PHP user functions. |
||||
- When adding an API, use a complete, recognizable snake_case name; do not use overly broad names such as `typephp_call()`. |
||||
- Functions used in only one `.cc` file should additionally be marked `static` or placed in an anonymous namespace. |
||||
- When crossing dynamic library boundaries, use the corresponding export macro; helpers that do not need to be exported should not widen symbol visibility. |
||||
- Do not use `typephp_` merely because the code is in `typephp_helper.h`; the criterion is whether the semantics are TypePHP-specific. |
||||
|
||||
### 2.2 Positive and Negative Examples |
||||
|
||||
```cpp |
||||
// Correct: the constructor chain semantics are TypePHP-specific. |
||||
typephp_call_parent_constructor(object, constructor, args); |
||||
|
||||
// Incorrect: this only materializes an INDIRECT zval into a plain value and is not TypePHP-specific. |
||||
typephp_deindirect(value); |
||||
|
||||
// Correct: generic Zend value wrapping belongs to PHPX. |
||||
php::deindirect(value); |
||||
``` |
||||
|
||||
## 3. `php::`: C++ Wrappers for ZendAPI |
||||
|
||||
`namespace php` is provided by PHPX to wrap Zend's C API, macros, raw pointers, and manual resource management into a type-safe, RAII-friendly C++ API. |
||||
|
||||
This naming domain contains two categories of capabilities: |
||||
|
||||
1. PHP values and runtime objects, such as `php::Var`, `php::Str`, `php::Array`, and `php::Object`; |
||||
2. Safe wrappers of ZendAPI, such as symbol lookup, scope management, value conversion, object creation, and invocation. |
||||
|
||||
Examples: |
||||
|
||||
```cpp |
||||
php::Var value; |
||||
php::Array arguments; |
||||
|
||||
auto plain = php::deindirect(value); |
||||
auto called_ce = php::getCalledCe(this_); |
||||
auto scope = php::getCallableScope(function, this_); |
||||
auto create_object = php::getCreateObjectFn(class_entry); |
||||
auto globals = php::globalsArray(); |
||||
``` |
||||
|
||||
### 3.1 When to Use `php::` |
||||
|
||||
Place into `namespace php` when the following conditions are met: |
||||
|
||||
- The API is meaningful to any PHPX C++ caller; |
||||
- The API's behavior can be fully explained by Zend/PHP runtime semantics; |
||||
- The API does not depend on TypePHP AST, compile-time Attributes, or TypePHP-specific language rules; |
||||
- The API's main purpose is to hide Zend macros, raw `zval *`, reference counting, or exception checking. |
||||
|
||||
### 3.2 Forbidding Global `php_*` Helpers |
||||
|
||||
The following legacy forms are forbidden: |
||||
|
||||
```cpp |
||||
php::Var php_deindirect(const php::Var &value); |
||||
php::Str php_get_called_class(php::Object &this_); |
||||
zend_class_entry *php_get_called_ce(php::Object &this_); |
||||
auto php_get_create_object_fn(zend_class_entry *ce); |
||||
``` |
||||
|
||||
They must be written as: |
||||
|
||||
```cpp |
||||
namespace php { |
||||
|
||||
Var deindirect(const Var &value); |
||||
Str getCalledClass(Object &this_); |
||||
zend_class_entry *getCalledCe(Object &this_); |
||||
auto getCreateObjectFn(zend_class_entry *ce); |
||||
|
||||
} // namespace php |
||||
``` |
||||
|
||||
The reason is that users can legitimately declare: |
||||
|
||||
```php |
||||
function deindirect(mixed $value): mixed {} |
||||
function get_called_ce(): string {} |
||||
function get_create_object_fn(): string {} |
||||
``` |
||||
|
||||
These PHP functions generate `php_deindirect`, `php_get_called_ce`, and |
||||
`php_get_create_object_fn`. If PHPX also defines same-named helpers globally, conflicts may occur at the declaration, overload resolution, or linking stage. |
||||
|
||||
### 3.3 Naming Style |
||||
|
||||
The PHPX C++ API uses the existing camelCase style: |
||||
|
||||
```cpp |
||||
php::getCalledClass(); |
||||
php::getClassEntrySafe(); |
||||
php::getPersistentCache(); |
||||
php::stdCreateObject(); |
||||
``` |
||||
|
||||
Do not mechanically preserve Zend's snake_case names as global C++ names. Lower-level calls can continue to use the original Zend API, such as `zend_objects_new()`, but the wrapper layer exposed to generated code should use `php::`. |
||||
|
||||
## 4. `typephp_<project>`: Project-private Namespace |
||||
|
||||
Each TypePHP compiled project has an independent C++ namespace: |
||||
|
||||
```text |
||||
typephp_<target-name> |
||||
``` |
||||
|
||||
For example, if the project name is `tpc`: |
||||
|
||||
```cpp |
||||
namespace typephp_tpc { |
||||
// Project-private generated state and helpers. |
||||
} |
||||
``` |
||||
|
||||
The `-` and `*` in the project name are converted to `_`, and the remaining characters must satisfy the compiler's target identifier validation. Because of the fixed `typephp_` prefix, the final C++ namespace is a valid identifier even if the project name starts with a digit. |
||||
|
||||
### 4.1 Content That Should Go into This Namespace |
||||
|
||||
- The literal string table and `get_str()`; |
||||
- The class/function/property cache tables and their accessor functions; |
||||
- Global variable storage of the current project; |
||||
- Class entries, object handlers, and default property templates; |
||||
- Module entry and MINIT/RINIT/RSHUTDOWN auxiliary state; |
||||
- Functions such as `module_init()` and `module_clean()` that are called only inside the generated extension file; |
||||
- Project-level generated state such as the Python module cache. |
||||
|
||||
Illustration: |
||||
|
||||
```cpp |
||||
namespace typephp_demo { |
||||
|
||||
static php::Str literal_strings[] = { |
||||
php::Str{"hello"}, |
||||
}; |
||||
|
||||
php::Str &get_str(uint32_t index) { |
||||
return literal_strings[index]; |
||||
} |
||||
|
||||
static THREAD_LOCAL zend_class_entry *class_map[8]; |
||||
|
||||
zend_class_entry *get_class(int id, const php::Str &name) { |
||||
// Resolve and cache a symbol owned by this project. |
||||
} |
||||
|
||||
static void module_init() { |
||||
// Initialize this project's generated state. |
||||
} |
||||
|
||||
} // namespace typephp_demo |
||||
``` |
||||
|
||||
### 4.2 Visibility and ABI |
||||
|
||||
- Names inside `typephp_<project>` are implementation details, not library stub ABI. |
||||
- Objects and functions that can be limited to `static` should continue to be marked `static`. |
||||
- Generated headers may declare project-internal accessors that must be used across translation units, but must not expose underlying arrays or cache tables. |
||||
- External handwritten C++ code must not depend on literal indexes, cache indexes, or project-internal storage names. |
||||
- Different TypePHP projects can be linked into the same process, because the same internal short names reside in different project namespaces. |
||||
|
||||
### 4.3 Scope Takes Priority over Name Spelling |
||||
|
||||
Historical generated names may still appear in the project namespace, for example: |
||||
|
||||
```cpp |
||||
typephp_demo::php_class_entry_App_User |
||||
``` |
||||
|
||||
Although the member name starts with `php_`, the full symbol resides in `typephp_demo`, so it is a project-private implementation rather than the global user callable ABI described in Section 5. New project-internal helpers should prefer short names without `php_`, such as `get_class()`, `get_func()`, and `get_str()`. |
||||
|
||||
## 5. `php_`: The C++ ABI of User PHP Callables |
||||
|
||||
The global `php_` prefix is used by TypePHP to map user-declared PHP functions and class methods into C++ callable symbols. This naming is used by generated code, library stubs, and external C++ implementations alike, so it cannot be changed arbitrarily. |
||||
|
||||
Example: |
||||
|
||||
```php |
||||
namespace App; |
||||
|
||||
function greet(string $name): string {} |
||||
|
||||
class User |
||||
{ |
||||
public function save(): bool {} |
||||
} |
||||
``` |
||||
|
||||
The conceptual C++ symbols are: |
||||
|
||||
```cpp |
||||
php::Str php_app__greet(php::Str name); |
||||
php::Bool php_app__user__save(php::Object &this_); |
||||
``` |
||||
|
||||
The rules include: |
||||
|
||||
- Use `php_` to mark "mapped from a PHP declaration"; |
||||
- PHP namespace, class, and method/function names are combined after normalization; |
||||
- `__` is the existing ABI combination separator; |
||||
- The first parameter of an instance method is the object `this_`; |
||||
- Stubs, libraries, and consumers must use exactly the same mapping rules. |
||||
|
||||
### 5.1 Why Internal Helpers Cannot Use `php_` |
||||
|
||||
The `php_` mapping is not an independent reserved keyword space, but a mechanical ABI of user PHP names. The following user declaration: |
||||
|
||||
```php |
||||
function deindirect(mixed $value): mixed {} |
||||
``` |
||||
|
||||
naturally generates: |
||||
|
||||
```cpp |
||||
php::Var php_deindirect(php::Var value); |
||||
``` |
||||
|
||||
Therefore, if the framework defines a global `php_deindirect()`, it encroaches on the user symbol space. The correct approach is `php::deindirect()`. |
||||
|
||||
### 5.2 Combination Collisions |
||||
|
||||
Because the current ABI uses `__` to combine PHP namespace, class, and callable names, the following two PHP declarations may map to the same C++ symbol: |
||||
|
||||
```php |
||||
function App\user__test(): void {} |
||||
|
||||
namespace App; |
||||
class User |
||||
{ |
||||
public function test(): void {} |
||||
} |
||||
``` |
||||
|
||||
The compiler must detect this situation during the preprocessing stage and throw a FatalError; it must not be handled through overriding, link order, or added runtime dispatch. Changing the mapping separator rules would break existing stubs/ABI, so collisions must be resolved by the user through renaming. |
||||
|
||||
### 5.3 Entry Symbol Exceptions |
||||
|
||||
A small number of C ABI/embed entry points are fixed by the generator and do not belong to ordinary user callables. For example: |
||||
|
||||
```cpp |
||||
php_<project>_embed_get_module(); |
||||
typephp_<project>_runtime_init(argc, argv); |
||||
typephp_<project>_runtime_shutdown(); |
||||
``` |
||||
|
||||
These are the connection points between the binary/library embed runtime and the current project's module entry. Definitions and references are uniformly generated through |
||||
`TYPEPHP_EMBED_GET_MODULE_FUNCTION()`, `TYPEPHP_RUNTIME_INIT_FUNCTION()`, |
||||
`TYPEPHP_RUNTIME_SHUTDOWN_FUNCTION()`, and the corresponding symbol macros, in a style consistent with Zend's |
||||
`PHP_MINIT_FUNCTION()`/`PHP_MINIT()`. The final symbols contain the project name and must not be used as a general helper naming template. |
||||
|
||||
### 5.4 The Shared Runtime in Multi-extension Processes |
||||
|
||||
TypePHP extensions must not separately compile or statically link a PHPX implementation containing process-level Zend state. The Reflection handler, |
||||
`FiberGenerator` class entry, scope, and Property Hook runtime are all provided solely by the shared `libphpx`: |
||||
|
||||
- Host-mode extensions/libraries must link `libphpx.so`, `libphpx.dylib`, or `phpx.dll`, and must not fall back to `libphpx.a`; |
||||
- Unix PHP extensions do not link the Embed `libphp.so`; Zend/PHP symbols are provided by the SAPI that loads them; |
||||
- macOS extensions use `-undefined dynamic_lookup` to resolve host symbols; |
||||
- Binaries and standalone WASI programs can still link statically, because each process or Wasm instance has only one copy of the runtime. |
||||
|
||||
`src/core/typephp_*.cc` only carries the TypePHP-specific `typephp_*` runtime; `php::` ZendAPI wrappers should be placed in core source files without the |
||||
`typephp_` prefix, such as `src/core/scope.cc`. |
||||
|
||||
## 6. Name Selection Flow |
||||
|
||||
When adding a C++ API, judge in the following order: |
||||
|
||||
1. **Is it the compiled body of a user PHP function or class method?** |
||||
- Yes: use the existing `php_` callable ABI generator; do not handwrite another mapping. |
||||
2. **Does it serve only one current TypePHP project?** |
||||
- Yes: place it in `typephp_<project>`, and use `static` or private accessors where possible. |
||||
3. **Does it implement TypePHP-specific semantics?** |
||||
- Yes: use the `typephp_` prefix. |
||||
4. **Is it only a C++ wrapper of Zend/PHP runtime capabilities?** |
||||
- Yes: place it in `namespace php`, using the PHPX camelCase style. |
||||
5. **None of the above?** |
||||
- It should not be arbitrarily added to `typephp_helper.h`; reconfirm its owning module and public API boundary. |
||||
|
||||
## 7. Code Review Checklist |
||||
|
||||
When adding or modifying generated helpers, check: |
||||
|
||||
- [ ] No new global `php_*` helpers in `typephp_helper.h`; |
||||
- [ ] ZendAPI wrappers are in `namespace php`; |
||||
- [ ] TypePHP-specific logic uses `typephp_`; |
||||
- [ ] Project caches and storage are in `typephp_<project>`; |
||||
- [ ] Project-private tables are not exposed directly via `extern` through generated headers; |
||||
- [ ] User callables still use the unified `php_` ABI generator; |
||||
- [ ] New names do not collide with user-declarable PHP functions or methods; |
||||
- [ ] bin, lib, ext, and WASM builds use the same project name derivation rule; |
||||
- [ ] Stub and existing ABI are evaluated together when modifying the public callable mapping; |
||||
- [ ] At least one compilation regression test is added for a user function with the same name. |
||||
|
||||
The current related regression test is: |
||||
|
||||
```text |
||||
tests/compiler/basic/helper-symbol-collision.phpt |
||||
``` |
||||
|
||||
## 8. Main Implementation Locations |
||||
|
||||
| Responsibility | File | |
||||
| --- | --- | |
||||
| `php_` callable prefix and combination separator | `src/CompilerBase.php` | |
||||
| Callable combination collision detection | `src/Preprocessor.php` | |
||||
| `typephp_<project>` generation and project-private tables | `src/Translator.php` | |
||||
| TypePHP extension prefix constants | `src/Metadata/Constants.php` | |
||||
| PHPX/TypePHP helper classification | `vendor/swoole/phpx/include/typephp_helper.h` | |
||||
| Embed module accessor concatenation | `vendor/swoole/phpx/src/misc/typephp_main.cc` | |
||||
@ -0,0 +1,99 @@ |
||||
# GMP Function Comparison Table (not implemented) |
||||
|
||||
This document records the functions of the PHP GMP extension that have not yet been implemented in the BigInt type, as a reference for future development. |
||||
|
||||
## Statistics |
||||
|
||||
The GMP extension has 44 functions in total (excluding `gmp_init` and the alias `gmp_div`). 17 are covered, 27 are not covered. |
||||
|
||||
## Implemented |
||||
|
||||
| GMP function | BigInt method / operator | Description | |
||||
|----------|---------------------|------| |
||||
| `gmp_init` | `std::bigInt()` | construction | |
||||
| `gmp_add` | `add()` / `+` | addition | |
||||
| `gmp_sub` | `sub()` / `-` | subtraction | |
||||
| `gmp_mul` | `mul()` / `*` | multiplication | |
||||
| `gmp_div_q` | `div()` / `/` | division (quotient) | |
||||
| `gmp_div_r` | `mod()` / `%` | division (remainder) | |
||||
| `gmp_div_qr` | `divmod()` | quotient and remainder | |
||||
| `gmp_mod` | `mod()` / `%` | modulo | |
||||
| `gmp_pow` | `pow()` | power | |
||||
| `gmp_powm` | `powmod()` | modular exponentiation | |
||||
| `gmp_neg` | `neg()` / `-` (unary) | negation | |
||||
| `gmp_abs` | `abs()` | absolute value | |
||||
| `gmp_sqrt` | `sqrt()` | square root | |
||||
| `gmp_gcd` | `gcd()` | greatest common divisor | |
||||
| `gmp_cmp` | `cmp()` / `<=>` | comparison | |
||||
| `gmp_and` | `bitAnd()` / `&` | bitwise AND | |
||||
| `gmp_or` | `bitOr()` / `\|` | bitwise OR | |
||||
| `gmp_xor` | `bitXor()` / `^` | bitwise XOR | |
||||
| `gmp_com` | `bitNot()` / `~` | bitwise NOT | |
||||
| `gmp_testbit` | `testBit()` | bit test | |
||||
| `gmp_popcount` | `popCount()` | population count | |
||||
| `gmp_intval` | `toInt()` | to int | |
||||
| `gmp_strval` | `toString()` | to string | |
||||
|
||||
## Not implemented (sorted by priority) |
||||
|
||||
### High priority — commonly used number-theory functions |
||||
|
||||
| GMP function | Suggested method name | Signature | Description | |
||||
|----------|-----------|------|------| |
||||
| `gmp_sign` | `sign()` | `(): int` | sign, returns -1/0/1 | |
||||
| `gmp_lcm` | `lcm($x)` | `(BigInt): BigInt` | least common multiple | |
||||
| `gmp_perfect_square` | `perfectSquare()` | `(): bool` | whether it is a perfect square | |
||||
| `gmp_perfect_power` | `perfectPower()` | `(): bool` | whether it is a perfect power | |
||||
| `gmp_prob_prime` | `probPrime($reps = 10)` | `(int): int` | probabilistic primality test (Miller-Rabin) | |
||||
| `gmp_nextprime` | `nextPrime()` | `(): BigInt` | next prime | |
||||
| `gmp_binomial` | `binomial($k)` | `(int): BigInt` | binomial coefficient C(n, k) | |
||||
| `gmp_fact` | `fact()` | `(): BigInt` | factorial n! | |
||||
|
||||
### Medium priority — advanced number-theory functions |
||||
|
||||
| GMP function | Suggested method name | Signature | Description | |
||||
|----------|-----------|------|------| |
||||
| `gmp_gcdext` | `gcdext($x)` | `(BigInt): array` | extended GCD, returns [g, s, t] such that g = s·a + t·b | |
||||
| `gmp_invert` | `invert($mod)` | `(BigInt): BigInt\|false` | modular inverse, returns false when it does not exist | |
||||
| `gmp_sqrtrem` | `sqrtrem()` | `(): array` | square root + remainder, returns [root, rem] | |
||||
| `gmp_jacobi` | `jacobi($x)` | `(BigInt): int` | Jacobi symbol | |
||||
| `gmp_legendre` | `legendre($x)` | `(BigInt): int` | Legendre symbol | |
||||
| `gmp_kronecker` | `kronecker($x)` | `(BigInt): int` | Kronecker symbol | |
||||
|
||||
### Low priority — less commonly used |
||||
|
||||
| GMP function | Suggested method name | Signature | Description | |
||||
|----------|-----------|------|------| |
||||
| `gmp_divexact` | `divExact($x)` | `(BigInt): BigInt` | exact division (used when divisibility is known; faster than ordinary division) | |
||||
| `gmp_root` | `root($n)` | `(int): BigInt` | n-th root (truncated) | |
||||
| `gmp_rootrem` | `rootrem($n)` | `(int): array` | n-th root + remainder | |
||||
| `gmp_hamdist` | `hamDist($x)` | `(BigInt): int` | Hamming distance | |
||||
|
||||
### Not applicable — conflicts with the immutable design |
||||
|
||||
| GMP function | Reason | |
||||
|----------|------| |
||||
| `gmp_setbit` | directly modifies the GMP object; BigInt is immutable | |
||||
| `gmp_clrbit` | directly modifies the GMP object; BigInt is immutable | |
||||
|
||||
### To be evaluated |
||||
|
||||
| GMP function | Description | |
||||
|----------|------| |
||||
| `gmp_scan0` | finds the first 0 bit from the specified position | |
||||
| `gmp_scan1` | finds the first 1 bit from the specified position | |
||||
| `gmp_random_bits` | generates a random-bit BigInt (needs a global seed; not suitable as an instance method) | |
||||
| `gmp_random_range` | random BigInt in a range (needs a global seed; not suitable as an instance method) | |
||||
| `gmp_random_seed` | sets the random seed (global state; not suitable as an instance method) | |
||||
| `gmp_import` | imports from a binary string | |
||||
| `gmp_export` | exports to a binary string | |
||||
|
||||
## toString enhancement |
||||
|
||||
| Missing feature | Description | |
||||
|---------|------| |
||||
| `toString($base)` | the current `toString()` only supports decimal. GMP's `gmp_strval` supports base 2-62 output | |
||||
|
||||
## Update log |
||||
|
||||
- 2026-05-27: initial version, compared against PHP 8.4.14 GMP extension |
||||
@ -0,0 +1,766 @@ |
||||
# AOT Compiler High-Precision Types Tutorial |
||||
|
||||
This tutorial introduces the three high-precision numeric types in the AOT compiler — **BigInt** (arbitrary-precision integer), **Decimal** (50-digit decimal number), and **BigFloat** (256-bit floating-point number). |
||||
|
||||
## Table of Contents |
||||
|
||||
1. [Why High-Precision Types Are Needed](#1-why-high-precision-types-are-needed) |
||||
2. [Quick Start](#2-quick-start) |
||||
3. [Overview of the Three Types](#3-overview-of-the-three-types) |
||||
4. [Construction and Declaration](#4-construction-and-declaration) |
||||
5. [Arithmetic Operations](#5-arithmetic-operations) |
||||
6. [Comparison Operations](#6-comparison-operations) |
||||
7. [Compound Assignment](#7-compound-assignment) |
||||
8. [Universal Method Calls](#8-universal-method-calls) |
||||
9. [Type Conversion](#9-type-conversion) |
||||
10. [Mixed Operations and Type Promotion](#10-mixed-operations-and-type-promotion) |
||||
11. [Automatic Detection of Extra-Long Literals](#11-automatic-detection-of-extra-long-literals) |
||||
12. [Limitations and Notes](#12-limitations-and-notes) |
||||
13. [Complete Examples](#13-complete-examples) |
||||
|
||||
--- |
||||
|
||||
## 1. Why High-Precision Types Are Needed |
||||
|
||||
PHP's native `int` is a 64-bit signed integer with a maximum value of `9223372036854775807` (about 9.22×10¹⁸). Integer literals exceeding this range are silently converted by the PHP parser to `float` (double), losing significant digits. |
||||
|
||||
PHP's native `float` (IEEE 754 double) can only guarantee about 15–16 significant digits at most. This is far from sufficient for financial computation, scientific computing, cryptography, and other scenarios. |
||||
|
||||
```php |
||||
// Precision problems with native PHP behavior |
||||
$a = 123456789012345678901234567890; // 30-digit integer → converted to float, precision lost |
||||
// Actually stored: 1.2345678901234568E+29, the trailing digits are already unreliable |
||||
|
||||
$b = 0.1 + 0.2; // 0.30000000000000004 — the classic floating-point error |
||||
``` |
||||
|
||||
The AOT compiler provides three high-precision types, built on mature C/C++ math libraries, and directly generates native calls. Here "zero-cost abstraction" refers to the absence of PHP method lookup and interpreter dispatch overhead; the high-precision operations themselves still require math library computation, memory allocation, and boxing: |
||||
|
||||
| Type | Underlying library | Characteristics | |
||||
|------|--------|------| |
||||
| BigInt | GMP (`libgmp`) | Arbitrary-precision integer, never overflows | |
||||
| Decimal | libmpdec | Decimal fraction, about 50 significant digits, no binary floating-point error | |
||||
| BigFloat | MPFR (`libmpfr`) | 256 bit by default, 64 significant digits in string output | |
||||
|
||||
--- |
||||
|
||||
## 2. Quick Start |
||||
|
||||
Prerequisites for using high-precision types: |
||||
|
||||
1. Declare `declare(strict_types=1)` at the top of the file |
||||
2. Import the native type declaration `use native_types` |
||||
3. The system must have the corresponding C++ libraries installed (`libgmp-dev`, `libmpdec-dev`, `libmpfr-dev`) |
||||
|
||||
```php |
||||
<?php |
||||
declare(strict_types=1); |
||||
use native_types; |
||||
|
||||
function main(): void { |
||||
// Your high-precision computation code |
||||
$a = std::bigInt("123456789012345678901234567890"); |
||||
$b = std::bigInt("987654321098765432109876543210"); |
||||
$sum = $a + $b; |
||||
echo $sum->toString(); |
||||
} |
||||
?> |
||||
``` |
||||
|
||||
Compile and run: |
||||
|
||||
```bash |
||||
php bin/tpc.php my_program.php -o my_program |
||||
./my_program |
||||
``` |
||||
|
||||
> **Tip**: Like all native_types, the Big* types can only be used in AOT compile mode and cannot run in the normal PHP interpreter. The AOT compiler performs compile-time evaluation of functions such as `std::bigInt()` and directly generates C++ code. |
||||
|
||||
--- |
||||
|
||||
## 3. Overview of the Three Types |
||||
|
||||
### BigInt — Arbitrary-Precision Integer |
||||
|
||||
Suitable for large integer computation; it never overflows and never loses precision. Integer division produces an integer result (truncated). |
||||
|
||||
```php |
||||
$a = std::bigInt("1234567890123456789012345678901234567890"); // 40 digits |
||||
$b = $a * 2; // 80 digits, never overflows |
||||
``` |
||||
|
||||
### Decimal — 50-Digit Decimal Number |
||||
|
||||
Suitable for scenarios requiring precise decimal representation, such as financial computation. `0.1 + 0.2` exactly equals `0.3` with no binary floating-point error. |
||||
|
||||
```php |
||||
$price = std::decimal("19.99"); |
||||
$quantity = 3; |
||||
$total = $price * $quantity; // 59.97, exact |
||||
``` |
||||
|
||||
### BigFloat — 256-Bit High-Precision Floating-Point Number |
||||
|
||||
Suitable for scenarios requiring high-precision floating-point computation, such as scientific computing. Based on MPFR, the default precision is currently fixed at 256 bit, far higher than the 53 bit of IEEE 754 double. |
||||
|
||||
```php |
||||
$pi = std::bigFloat("3.141592653589793238462643383279502884197"); |
||||
$area = $pi * 100 * 100; // high-precision π × r² |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 4. Construction and Declaration |
||||
|
||||
### 4.1 Constructing from Literals |
||||
|
||||
`std::bigInt()`, `std::decimal()`, and `std::bigFloat()` are **compile-time functions** that directly construct the corresponding C++ objects in the generated C++ code, without producing runtime function calls. |
||||
|
||||
```php |
||||
// BigInt — construct from an int or a string |
||||
$a = std::bigInt(100); // ordinary integer |
||||
$b = std::bigInt("123456789012345678901234567890"); // extra-long integer, must use a string |
||||
|
||||
// Decimal — construct from a string is recommended to avoid floating-point precision loss |
||||
$c = std::decimal("123.456"); // ✅ recommended: exact string |
||||
$d = std::decimal(42); // ✅ acceptable: from int |
||||
|
||||
// BigFloat — construct from an int, float, or string |
||||
$e = std::bigFloat(100.5); // from float |
||||
$f = std::bigFloat(42); // from int |
||||
$g = std::bigFloat("3.14159265358979323846"); // from string (exact) |
||||
``` |
||||
|
||||
### 4.2 Type Annotation |
||||
|
||||
Under `use native_types`, Big* type variables automatically get native C++ storage types: |
||||
|
||||
```php |
||||
use native_types; |
||||
|
||||
// The compiler automatically infers the type as php::BigInt / php::Decimal / php::BigFloat |
||||
$a = std::bigInt(100); // → C++: php::Variant(new BigInt(100)) |
||||
$b = std::decimal("100.50"); // → C++: php::Variant(new Decimal("100.50")) |
||||
$c = std::bigFloat(3.14); // → C++: php::Variant(new BigFloat(3.14)) |
||||
``` |
||||
|
||||
> **Key detail**: Big* types are **immutable**. Every operation returns a new value and never modifies the original variable. See [Section 7: Compound Assignment](#7-compound-assignment) for details. |
||||
|
||||
--- |
||||
|
||||
## 5. Arithmetic Operations |
||||
|
||||
### 5.1 Standard Operators |
||||
|
||||
The supported operators depend on the concrete type: BigInt supports `+ - * / % **`, Decimal supports `+ - * / %`, and BigFloat supports `+ - * /`: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$b = std::bigInt(200); |
||||
|
||||
$sum = $a + $b; // addition |
||||
$diff = $a - $b; // subtraction |
||||
$prod = $a * $b; // multiplication |
||||
$quot = $a / $b; // division (integer division for BigInt) |
||||
$mod = $a % $b; // modulo |
||||
$pow = $a ** 10; // exponentiation (supported by BigInt) |
||||
|
||||
// unary negation |
||||
$neg = -$a; // negation |
||||
``` |
||||
|
||||
Example of the generated C++ code (`$a + $b`): |
||||
|
||||
```cpp |
||||
php::BigInt::add(a, b) // BigInt addition |
||||
php::BigInt::sub(a, b) // BigInt subtraction |
||||
php::BigInt::mul(a, b) // BigInt multiplication |
||||
php::BigInt::div(a, b) // BigInt division |
||||
php::BigInt::mod(a, b) // BigInt modulo |
||||
php::BigInt::pow(a, b) // BigInt exponentiation |
||||
``` |
||||
|
||||
### 5.2 Mixed Operations with int / float |
||||
|
||||
Big* types can be mixed with ordinary int/float within a safe range, and the compiler automatically performs type promotion: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
|
||||
$b = $a + 50; // BigInt + Int → BigInt |
||||
$c = 200 + $a; // Int + BigInt → BigInt |
||||
$d = $a * 3.5; // BigInt * Float → compile error! |
||||
// a float cannot be promoted to BigInt exactly, |
||||
// use Decimal or BigFloat instead |
||||
``` |
||||
|
||||
### 5.3 BigInt Division Notes |
||||
|
||||
`BigInt / BigInt` is integer division (truncation), similar to PHP's `intdiv()`: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$b = $a / 3; // 33 (not 33.333...) |
||||
``` |
||||
|
||||
If you need an exact decimal result, convert the operands to Decimal first: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$result = std::decimal($a->toString()) / std::decimal("3"); |
||||
// 33.333333333... |
||||
``` |
||||
|
||||
### 5.4 Summary of Operators Supported by Each Type |
||||
|
||||
| Operator | BigInt | Decimal | BigFloat | |
||||
|--------|--------|---------|----------| |
||||
| `+` `-` `*` | ✅ | ✅ | ✅ | |
||||
| `/` | ✅ integer division | ✅ | ✅ | |
||||
| `%` | ✅ | ✅ | ❌ | |
||||
| `**` | ✅ | ❌ | ❌ | |
||||
| `-` (unary negation) | ✅ | ✅ | ✅ | |
||||
|
||||
--- |
||||
|
||||
## 6. Comparison Operations |
||||
|
||||
All six comparison operators can be used with Big* types: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$b = 200; |
||||
|
||||
// comparison operations return bool (an (int) cast is needed for output) |
||||
echo (int)($a < $b); // 1 (true) → cmp(a,b) < 0 |
||||
echo (int)($a > $b); // 0 (false) → cmp(a,b) > 0 |
||||
echo (int)($a <= 100); // 1 (true) → cmp(a,b) <= 0 |
||||
echo (int)($a >= 100); // 1 (true) → cmp(a,b) >= 0 |
||||
echo (int)($a == 100); // 1 (true) → cmp(a,b) == 0 |
||||
echo (int)($a != 50); // 1 (true) → cmp(a,b) != 0 |
||||
|
||||
// spaceship operator |
||||
$cmp = $a <=> $b; // -1 ($a < $b) |
||||
echo (int)$cmp; // -1 |
||||
``` |
||||
|
||||
Example of the generated C++ code: |
||||
|
||||
```cpp |
||||
php::BigInt::cmp(a, b) < 0 // a < b |
||||
php::BigInt::cmp(a, b) == 0 // a == b |
||||
php::BigInt::cmp(a, b) != 0 // a != b |
||||
php::BigInt::cmp(a, b) // a <=> b (directly returns -1/0/1) |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 7. Compound Assignment |
||||
|
||||
Big* types **support** compound assignment operators such as `+=`, `-=`, `*=`, `/=`, and `%=`. |
||||
|
||||
### 7.1 How It Works |
||||
|
||||
Big* types are **immutable**. `$a += 50` is expanded at compile time to `$a = BigInt::add($a, 50)` — a new value is created and then assigned to the original variable. |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$a += 50; // → a = php::BigInt::add(a, php::newBigInt(50)) |
||||
echo $a->toString(); // "150" |
||||
|
||||
$a -= 30; // → a = php::BigInt::sub(a, php::newBigInt(30)) |
||||
$a *= 5; // → a = php::BigInt::mul(a, php::newBigInt(5)) |
||||
$a /= 3; // → a = php::BigInt::div(a, php::newBigInt(3)) |
||||
$a %= 7; // → a = php::BigInt::mod(a, php::newBigInt(7)) |
||||
``` |
||||
|
||||
Decimal and BigFloat support it likewise: |
||||
|
||||
```php |
||||
// Decimal compound assignment |
||||
$d = std::decimal("100.50"); |
||||
$d += 25.25; // → d = php::Decimal::add(d, php::newDecimal("25.25")) |
||||
$d -= 123.45; // → d = php::Decimal::sub(d, php::newDecimal("123.45")) |
||||
$d *= 2; // → d = php::Decimal::mul(d, php::newDecimal(2)) |
||||
$d /= 4; // → d = php::Decimal::div(d, php::newDecimal(4)) |
||||
$d %= 5.0; // → d = php::Decimal::mod(d, php::newDecimal("5.0")) |
||||
|
||||
// BigFloat compound assignment (% is not supported) |
||||
$bf = std::bigFloat(100.0); |
||||
$bf += 50.0; |
||||
$bf -= 30.0; |
||||
$bf *= 2.0; |
||||
$bf /= 3.0; |
||||
``` |
||||
|
||||
### 7.2 `++` / `--` Are Unavailable |
||||
|
||||
Because Big* types are immutable, the `++` / `--` operators do not match semantically. The compiler gives a clear error message: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$a++; // ❌ compile error: Cannot use ++ on php::BigInt. Use += 1 instead. |
||||
++$a; // ❌ compile error: Cannot use ++ on php::BigInt. Use += 1 instead. |
||||
--$a; // ❌ compile error: Cannot use -- on php::BigInt. Use -= 1 instead. |
||||
``` |
||||
|
||||
The correct alternatives: |
||||
|
||||
```php |
||||
$a += 1; // ✅ instead of $a++ |
||||
$a -= 1; // ✅ instead of $a-- |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 8. Universal Method Calls |
||||
|
||||
Big* types support calling methods via the `$value->method()` syntax (Universal Methods). These calls are directly translated at compile time to the corresponding C++ static functions, with no dynamic method dispatch overhead; the math library computation, result allocation, and boxing costs still remain. |
||||
|
||||
### 8.1 BigInt Methods |
||||
|
||||
```php |
||||
$a = std::bigInt("12345678901234567890"); |
||||
|
||||
// arithmetic methods (all return a new BigInt) |
||||
$b = $a->add(1); // addition: $a + 1 |
||||
$c = $a->sub(1); // subtraction: $a - 1 |
||||
$d = $a->mul(2); // multiplication: $a * 2 |
||||
$e = $a->div(10); // division: $a / 10 |
||||
$f = $a->mod(1000000); // modulo: $a % 1000000 |
||||
$g = $a->pow(3); // exponentiation: $a ** 3 |
||||
|
||||
// unary methods |
||||
$h = $a->neg(); // negation: -$a |
||||
$i = $a->abs(); // absolute value |
||||
|
||||
// special methods |
||||
$j = $a->gcd(15); // greatest common divisor: gcd($a, 15) |
||||
|
||||
// comparison methods |
||||
$cmp = $a->cmp(100); // comparison: returns -1/0/1 |
||||
if ($a->cmp(100) > 0) { /* $a > 100 */ } |
||||
|
||||
// type conversion methods |
||||
echo $a->toString(); // to string: "12345678901234567890" |
||||
echo $a->toInt(); // to int; throws ArithmeticError when out of the PHP int range |
||||
echo $a->toFloat(); // to float (may lose precision) |
||||
``` |
||||
|
||||
### 8.2 Decimal Methods |
||||
|
||||
```php |
||||
$d = std::decimal("123.456"); |
||||
|
||||
// arithmetic methods |
||||
echo $d->add(std::decimal("50.25"))->toString(); // "173.706" |
||||
echo $d->sub(std::decimal("50.25"))->toString(); // "73.206" |
||||
echo $d->mul(2)->toString(); // "246.912" |
||||
echo $d->div(3)->toString(); // "41.152" |
||||
echo $d->mod(std::decimal("5.0"))->toString(); // "3.456" |
||||
|
||||
// unary methods |
||||
echo $d->neg()->toString(); // "-123.456" |
||||
echo $d->abs()->toString(); // "123.456" |
||||
|
||||
// comparison and conversion |
||||
echo $d->cmp(std::decimal("100")) > 0 ? "greater" : "less"; // "greater" |
||||
echo $d->toInt(); // 123 |
||||
echo $d->toString(); // "123.456" |
||||
``` |
||||
|
||||
### 8.3 BigFloat Methods |
||||
|
||||
```php |
||||
$bf = std::bigFloat(3.14159265); |
||||
|
||||
echo $bf->add(1.0)->toString(); // "4.14159265..." |
||||
echo $bf->mul(2.0)->toString(); // "6.2831853..." |
||||
echo $bf->div(2.0)->toFloat(); // 1.570796325 |
||||
echo $bf->neg()->toString(); // "-3.14159265..." |
||||
echo $bf->abs()->toString(); // "3.14159265..." |
||||
|
||||
// comparison |
||||
echo $bf->cmp(3.0); // > 0 ($bf > 3.0) |
||||
``` |
||||
|
||||
### 8.4 Universal Methods vs Operators |
||||
|
||||
Operators and method calls are functionally equivalent; which one to choose depends on coding style: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$b = std::bigInt(50); |
||||
|
||||
// two equivalent ways of writing |
||||
$result1 = $a + $b; // operator style |
||||
$result2 = $a->add($b); // method call style |
||||
|
||||
// method calls support chaining |
||||
$result3 = $a->add(10)->mul(2)->sub(5)->toString(); // "215" |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 9. Type Conversion |
||||
|
||||
### 9.1 Conversion Between Big* Types |
||||
|
||||
```php |
||||
// BigInt → Decimal (exact, recommended approach) |
||||
$big = std::bigInt("12345678901234567890"); |
||||
$dec = std::decimal($big->toString()); |
||||
|
||||
// Decimal → BigInt (truncates the fractional part) |
||||
$d = std::decimal("123.456"); |
||||
$i = std::bigInt($d->toInt()); // 123 |
||||
|
||||
// Int → BigInt / Decimal / BigFloat |
||||
$bi = std::bigInt(42); |
||||
$dc = std::decimal(42); |
||||
$bf = std::bigFloat(42); |
||||
|
||||
// Float → BigFloat (using a float literal directly for Float → Decimal is not recommended) |
||||
$bf2 = std::bigFloat(3.14); |
||||
|
||||
// any type → BigFloat |
||||
$bf3 = std::bigFloat($big->toString()); |
||||
``` |
||||
|
||||
### 9.2 Conversion Between Big* and Ordinary Types |
||||
|
||||
```php |
||||
// BigInt → ordinary types |
||||
$a = std::bigInt("99999999999999999999"); |
||||
$s = $a->toString(); // "99999999999999999999" |
||||
$i = $a->toInt(); // throws ArithmeticError when out of the PHP int range |
||||
$f = $a->toFloat(); // 1.0E+20 (may lose precision) |
||||
|
||||
// ordinary types → BigInt (via compile-time functions) |
||||
$b = std::bigInt(42); // int → BigInt |
||||
$c = std::bigInt("123456..."); // string → BigInt |
||||
|
||||
// explicit casts and PHP conversion functions convert numerically and do not read the Box resource id |
||||
$n = (int) std::decimal("12.75"); // 12 |
||||
$x = floatval(std::bigInt("42")); // 42.0 |
||||
$ok = boolval(std::bigFloat("0")); // false |
||||
``` |
||||
|
||||
### 9.3 Limitations on Cross-Type Implicit Mixing |
||||
|
||||
The compiler blocks cross-type implicit mixing operations that may cause precision loss: |
||||
|
||||
```php |
||||
$a = std::bigFloat(100.5); |
||||
$b = std::bigInt(200); |
||||
|
||||
$c = $a + $b; // ❌ compile error: Cannot mix BigFloat and BigInt implicitly. |
||||
// Use std::bigFloat() to convert explicitly. |
||||
|
||||
// the correct approach: explicit conversion |
||||
$c = $a + std::bigFloat($b->toString()); // ✅ |
||||
``` |
||||
|
||||
| Combination | Allowed | Description | |
||||
|------|---------|------| |
||||
| BigInt + BigFloat | ❌ compile error | different precision metrics, explicit conversion required | |
||||
| BigInt + Decimal | ❌ compile error | different precision metrics, explicit conversion required | |
||||
| BigFloat + Decimal | ❌ compile error | different precision metrics, explicit conversion required | |
||||
| BigInt + Int | ✅ automatically promote Int → BigInt | no precision loss | |
||||
| BigInt + Float | ❌ compile error | Float cannot be promoted to BigInt exactly | |
||||
| Decimal + Int | ✅ automatically promote Int → Decimal | no precision loss | |
||||
| Decimal + Float | ✅ automatically promote Float → Decimal | may have a tiny error | |
||||
| BigFloat + Int | ✅ automatically promote Int → BigFloat | no precision loss | |
||||
| BigFloat + Float | ✅ automatically promote Float → BigFloat | no precision loss | |
||||
|
||||
--- |
||||
|
||||
## 10. Mixed Operations and Type Promotion |
||||
|
||||
When Big* types are mixed with ordinary Int/Float, the compiler only performs safe promotions that do not change the numeric model. |
||||
|
||||
**Rules**: |
||||
|
||||
1. If either operand is a Var (non-native type), both are converted to Var and runtime computation uses the ZendVM |
||||
2. If both operands are Int/Float, Float takes precedence (Int → Float) |
||||
3. BigInt can safely promote Int; Decimal can promote Int and Float literals whose source text is preserved; BigFloat can promote Int/Float |
||||
4. No implicit conversion is performed between different Big* types, or between BigInt and Float |
||||
|
||||
```php |
||||
// type promotion examples |
||||
$a = std::bigInt(100); |
||||
$b = 50; // Int |
||||
|
||||
$c = $a + $b; // BigInt + Int → BigInt |
||||
// $b is automatically promoted to BigInt |
||||
|
||||
$d = std::decimal("10.5"); |
||||
$e = $d + 3; // Decimal + Int → Decimal |
||||
// 3 is automatically promoted to Decimal |
||||
|
||||
$f = std::bigFloat(1.5); |
||||
$g = $f + 2.0; // BigFloat + Float → BigFloat |
||||
// 2.0 is automatically promoted to BigFloat |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 11. Automatic Detection of Extra-Long Literals |
||||
|
||||
The AOT compiler automatically detects numeric literals that exceed the precision of native types and automatically converts them to the corresponding Big* type. You do **not need to wrap them manually**. |
||||
|
||||
```php |
||||
// integer with 19 or more digits → automatically converted to BigInt |
||||
$a = 12345678901234567890; |
||||
echo $a->toString(); // "12345678901234567890" |
||||
// the compiler handles it automatically: equivalent to std::bigInt("12345678901234567890") |
||||
|
||||
// decimal with 16 or more significant digits → automatically converted to Decimal |
||||
$b = 3.14159265358979323846; |
||||
// the compiler handles it automatically: equivalent to std::decimal("3.14159265358979323846") |
||||
``` |
||||
|
||||
**Detection rules**: |
||||
|
||||
- pure digits, 19 or more digits → BigInt |
||||
- contains a decimal point or exponent, 16 or more significant digits → Decimal |
||||
- underscores `_` are disabled (e.g. `1_234_567_890_123_456_789_0`) |
||||
|
||||
> **Recommended practice**: For critical precision, it is still recommended to explicitly use `std::bigInt("...")` or `std::decimal("...")` to ensure clear intent. Automatic detection is a convenience feature suited for rapid prototyping. |
||||
|
||||
--- |
||||
|
||||
## 12. Limitations and Notes |
||||
|
||||
### 12.1 Immutability |
||||
|
||||
All Big* types are **immutable**. Every operation creates a new value: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$b = $a->add(50); // $a is still 100, $b is 150 |
||||
$c = $a + 50; // $a is still 100, $c is 150 |
||||
``` |
||||
|
||||
### 12.2 `++` / `--` Not Supported |
||||
|
||||
See [Section 7.2](#72---are-unavailable). Use `+= 1` / `-= 1` instead. |
||||
|
||||
### 12.3 BigFloat Does Not Support `%` and `**` |
||||
|
||||
```php |
||||
$bf = std::bigFloat(10.0); |
||||
$bf %= 3; // ❌ compile error |
||||
$bf ** 2; // ❌ compile error |
||||
``` |
||||
|
||||
### 12.4 Decimal Does Not Support `**` |
||||
|
||||
```php |
||||
$d = std::decimal("10.5"); |
||||
$d ** 2; // ❌ compile error |
||||
``` |
||||
|
||||
### 12.5 Cross Big* Types Cannot Be Implicitly Mixed |
||||
|
||||
BigFloat, Decimal, and BigInt must be explicitly converted: |
||||
|
||||
```php |
||||
$a = std::bigFloat(100.5); |
||||
$b = std::bigInt(200); |
||||
$c = $a + $b; // ❌ compile error |
||||
// change to |
||||
$c = $a + std::bigFloat($b->toString()); // ✅ |
||||
``` |
||||
|
||||
This restriction also applies to comparison operations. Before comparing, both sides must be explicitly converted to the same Big* type to avoid compiling to the wrong underlying resource type. |
||||
|
||||
### 12.6 Boundaries and Exceptions |
||||
|
||||
- BigInt negative right shifts use arithmetic right shift, for example `std::bigInt("-3") >> 1` yields `-2`. |
||||
- A negative bit index, negative `popCount()`, or an excessively large exponent throws `ValueError`. |
||||
- Division by zero throws `DivisionByZeroError`; converting to a PHP int beyond range throws `ArithmeticError`. |
||||
- When the absolute value of a BigFloat's exponent exceeds 10000, `toString()` automatically uses scientific notation to avoid constructing an excessively large string. |
||||
|
||||
### 12.7 Cannot Run in the Normal PHP Interpreter |
||||
|
||||
Big* types are a proprietary feature of the AOT compiler, relying on compile-time code generation and C++ underlying libraries. The source code cannot be directly interpreted and executed by the `php` command. |
||||
|
||||
### 12.8 Enabling `use native_types` |
||||
|
||||
Forgetting to add `use native_types` causes Big* variables to be treated as Var (generic type), losing most of the performance advantages of native types. |
||||
|
||||
--- |
||||
|
||||
## 13. Complete Examples |
||||
|
||||
### 13.1 Large Integer Factorial |
||||
|
||||
```php |
||||
<?php |
||||
declare(strict_types=1); |
||||
use native_types; |
||||
|
||||
/** |
||||
* Compute the factorial of n, supporting arbitrarily large results |
||||
*/ |
||||
function factorial(int $n): void { |
||||
$result = std::bigInt(1); |
||||
for ($i = 2; $i <= $n; $i++) { |
||||
$result *= $i; |
||||
} |
||||
echo "{$n}! = " . $result->toString() . "\n"; |
||||
echo "digits: " . strlen($result->toString()) . "\n"; |
||||
} |
||||
|
||||
function main(): void { |
||||
factorial(10); // 10! = 3628800 |
||||
factorial(50); // 3041409320171337804361260816606476884... |
||||
factorial(100); // 933262154439441526816992388562667004... |
||||
} |
||||
?> |
||||
``` |
||||
|
||||
### 13.2 Financial Computation: Order Details |
||||
|
||||
```php |
||||
<?php |
||||
declare(strict_types=1); |
||||
use native_types; |
||||
|
||||
function main(): void { |
||||
// use Decimal to represent amounts exactly |
||||
$price = std::decimal("19.99"); |
||||
$quantity = 3; |
||||
$taxRate = std::decimal("0.08"); |
||||
|
||||
$subtotal = $price * $quantity; |
||||
$tax = $subtotal * $taxRate; |
||||
$total = $subtotal + $tax; |
||||
|
||||
echo "unit price: " . $price->toString() . "\n"; |
||||
echo "quantity: {$quantity}\n"; |
||||
echo "subtotal: " . $subtotal->toString() . "\n"; |
||||
echo "tax: " . $tax->toString() . "\n"; |
||||
echo "total: " . $total->toString() . "\n"; |
||||
} |
||||
?> |
||||
``` |
||||
|
||||
Output: |
||||
|
||||
``` |
||||
unit price: 19.99 |
||||
quantity: 3 |
||||
subtotal: 59.97 |
||||
tax: 4.7976 |
||||
total: 64.7676 |
||||
``` |
||||
|
||||
### 13.3 High-Precision Pi Computation |
||||
|
||||
```php |
||||
<?php |
||||
declare(strict_types=1); |
||||
use native_types; |
||||
|
||||
function main(): void { |
||||
// use BigFloat for high-precision math computation |
||||
$pi = std::bigFloat("3.141592653589793238462643383279502884197"); |
||||
$radius = 100; |
||||
|
||||
// area of a circle |
||||
$area = $pi * std::bigFloat($radius * $radius); |
||||
echo "circle area: " . $area->toString() . "\n"; |
||||
|
||||
// circumference of a circle |
||||
$circumference = $pi * std::bigFloat(2 * $radius); |
||||
echo "circumference: " . $circumference->toString() . "\n"; |
||||
|
||||
// comparison |
||||
$earthRadius = 6371; |
||||
$earthArea = $pi * std::bigFloat($earthRadius * $earthRadius); |
||||
echo "if the radius is {$earthRadius}km...\n"; |
||||
echo "approximate area: " . $earthArea->toInt() . " km²\n"; |
||||
} |
||||
?> |
||||
``` |
||||
|
||||
### 13.4 Comprehensive Example: Mixing Multiple Types |
||||
|
||||
```php |
||||
<?php |
||||
declare(strict_types=1); |
||||
use native_types; |
||||
|
||||
function main(): void { |
||||
// BigInt — large integer operations |
||||
$big = std::bigInt("100000000000000000000"); |
||||
$big += std::bigInt("99999999999999999999"); |
||||
echo "BigInt: " . $big->toString() . "\n"; |
||||
|
||||
// operators + comparison |
||||
$a = std::bigInt(100); |
||||
echo "BigInt + Int: " . ($a + 50)->toString() . "\n"; |
||||
echo "BigInt * 5: " . ($a * 5)->toString() . "\n"; |
||||
echo "BigInt > 50: " . (int)($a > 50) . "\n"; |
||||
echo "a == 100: " . (int)($a == 100) . "\n"; |
||||
|
||||
// Unary minus |
||||
$neg = -$a; |
||||
echo "-a: " . $neg->toString() . "\n"; |
||||
|
||||
// Decimal — exact decimal operations |
||||
$price = std::decimal("99.99"); |
||||
$price *= 3; // compound assignment |
||||
echo "price × 3: " . $price->toString() . "\n"; |
||||
|
||||
// comparison |
||||
$d = std::decimal("100.25"); |
||||
echo "d > 50: " . (int)($d > 50) . "\n"; |
||||
echo "d != 100: " . (int)($d != 100) . "\n"; |
||||
|
||||
// BigFloat — high-precision floating point |
||||
$bf = std::bigFloat(3.14159); |
||||
$bf *= 2.0; |
||||
echo "pi × 2: " . $bf->toString() . "\n"; |
||||
|
||||
// method chaining |
||||
$result = std::bigInt(100) |
||||
->add(50) |
||||
->mul(3) |
||||
->sub(100) |
||||
->toString(); |
||||
echo "100 + 50 × 3 - 100 = " . $result . "\n"; |
||||
} |
||||
?> |
||||
``` |
||||
|
||||
Output: |
||||
|
||||
``` |
||||
BigInt: 200000000000000000099 |
||||
BigInt + Int: 150 |
||||
BigInt * 5: 500 |
||||
BigInt > 50: 1 |
||||
a == 100: 1 |
||||
-a: -100 |
||||
price × 3: 299.97 |
||||
d > 50: 1 |
||||
d != 100: 1 |
||||
pi × 2: 6.2831800000000000 |
||||
100 + 50 × 3 - 100 = 350 |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Further Reading |
||||
|
||||
- **Type System Specification**: [`docs/NATIVE_TYPES.md`](NATIVE_TYPES.md) — complete type promotion rules, declaration syntax, and C++ API reference |
||||
- **BigInt PHPT Tests**: [`tests/compiler/bigint/`](../tests/compiler/bigint/) — integration tests for BigInt features |
||||
- **Decimal PHPT Tests**: [`tests/compiler/decimal/`](../tests/compiler/decimal/) — integration tests for Decimal features |
||||
- **BigFloat Integration Tests**: [`tests/compiler/bignumber/bigfloat_operators.phpt`](../tests/compiler/bignumber/bigfloat_operators.phpt) — BigFloat operator tests |
||||
- **C++ Runtime Header Files**: |
||||
- [`phpx/include/phpx_big_int.h`](../../phpx/include/phpx_big_int.h) — BigInt C++ API |
||||
- [`phpx/include/phpx_decimal.h`](../../phpx/include/phpx_decimal.h) — Decimal C++ API |
||||
- [`phpx/include/phpx_big_float.h`](../../phpx/include/phpx_big_float.h) — BigFloat C++ API |
||||
@ -0,0 +1,116 @@ |
||||
# Interface Property Hooks Implementation Plan |
||||
|
||||
This document records the design and implementation plan for TP-AOT-010. The goal is to support the PHP 8.4 Interface Property Hook contract while preserving the zero-cost abstraction of TypePHP Native calls, and to give the PHP 8.4 ZendVM complete metadata for Reflection, dynamic class linking, and inheritance checks. |
||||
|
||||
## Current status (2026-08-14) |
||||
|
||||
The first stage has landed: the Interface contract model, AOT implementation checks, get/set direction variance, PHPX abstract Hook metadata, Reflection, dynamic PHP implementation classes, and regression tests are all wired up. Explicit setter parameter types are still rejected at compile time per the convention below; they will be opened up once the independent write-type model is completed. |
||||
|
||||
## 1. Design conclusion |
||||
|
||||
A Hooked Property in an Interface only represents a property contract: |
||||
|
||||
```php |
||||
interface Named |
||||
{ |
||||
public string $name { get; set; } |
||||
} |
||||
``` |
||||
|
||||
- The Interface holds no property slot, generates no getter/setter implementation, and produces no contract check at access time. |
||||
- TypePHP verifies at compile time whether known AOT classes satisfy the property's visibility, type, and `get`/`set` capabilities. |
||||
- The PHP 8.4 target registers native Zend Hook metadata in MINIT so Reflection and dynamic PHP classes obtain the same contract. |
||||
- The minimum version of TypePHP, PHPX, and the final target runtime is PHP 8.4; no downgrade path is provided for older versions. |
||||
|
||||
## 2. Syntax and diagnostics |
||||
|
||||
Three kinds of contracts are supported: |
||||
|
||||
```php |
||||
public string $readable { get; } |
||||
public string $writable { set; } |
||||
public string $readWrite { get; set; } |
||||
``` |
||||
|
||||
An Interface Property Hook must be `public`, non-`static`, have no default value, and its Hooks must not contain function bodies. Ordinary Interface properties, `private`/`protected`, `readonly`, duplicate or unknown Hooks, and Hooks with implementation bodies all throw a FatalError at TypePHP compile time. Error messages should match PHP 8.4 as closely as possible. |
||||
|
||||
The first stage only accepts the implicit setter parameter: |
||||
|
||||
```php |
||||
public string $name { set; } |
||||
``` |
||||
|
||||
PHP 8.4 also allows explicit, contravariant setter parameters such as `set(string|Stringable $value)`. That syntax requires the compile-time contract model and the Zend Hook `arg_info` to simultaneously store a write type independent of the property read type; until this part is complete, TypePHP reports a clear compile-time error and does not generate potentially incorrect runtime metadata. |
||||
|
||||
## 3. Compiler model |
||||
|
||||
An Interface Property Hook must not be disguised as an ordinary property or as an ordinary method after lowering. A separate contract model is established, storing at least: |
||||
|
||||
- the property name and declaration node; |
||||
- the resolved TypePHP type and class type; |
||||
- whether `get` is required; |
||||
- whether `set` is required; |
||||
- visibility and other flags used for diagnostics. |
||||
|
||||
The contract is stored in `InterfaceDef`. The AST/preprocessing stage only collects and validates declarations; it does not allocate property slots for Interfaces, does not run the `PropertyHookLowering` used by concrete classes, and does not generate hidden methods. |
||||
|
||||
Contract linking is performed after all types finish preprocessing: parent Interface contracts are expanded, then the properties provided by the implementing class itself or its parent are checked. An ordinary public backed property satisfies both the read and write contracts; a Hooked Property is judged by its actual Hook capabilities. get-only types are covariant in the read direction, set-only types are contravariant in the write direction, and types containing both get and set remain invariant. |
||||
|
||||
## 4. PHPX and Zend metadata |
||||
|
||||
The existing `typephp_register_property_hooks()` is for concrete classes with real AOT getters/setters and cannot be reused for abstract Interface Hooks. |
||||
|
||||
PHPX adds a separate helper: |
||||
|
||||
```cpp |
||||
typephp_register_abstract_property_hooks( |
||||
zend_class_entry *interface_ce, |
||||
zend_property_info *property_info, |
||||
bool readable, |
||||
bool writable |
||||
); |
||||
``` |
||||
|
||||
TypePHP/PHPX already uniformly require PHP 8.4+, so this helper directly accesses the PHP 8.4 ABI and is responsible for: |
||||
|
||||
- persistently allocating `zend_property_info::hooks`; |
||||
- creating abstract `get`/`set` `zend_internal_function` metadata without handlers; |
||||
- setting `ZEND_ACC_PUBLIC | ZEND_ACC_ABSTRACT`, the correct parameter/return types, and `common.prop_info`; |
||||
- updating `num_hooked_props` so Zend inheritance and Reflection recognize the contract; |
||||
- ensuring all strings, Hook tables, and function descriptors have MINIT-level persistent lifetimes. |
||||
|
||||
The generated code first registers the Interface, then declares the property with `IS_UNDEF` and `ZEND_ACC_PUBLIC | ZEND_ACC_ABSTRACT | ZEND_ACC_VIRTUAL` and mounts the abstract Hooks, and finally registers and links the implementing classes. |
||||
|
||||
## 5. PHP version boundary |
||||
|
||||
TypePHP distinguishes the source language version from the linked runtime: |
||||
|
||||
- `--php-version` allows only `8.4` or `8.5` and is used to parse syntax and handle project conditions; |
||||
- PHPX headers, `libphp`, and the final runtime must be PHP 8.4 or higher; |
||||
- `--php-version` and `libphp.so` are not required to have exactly matching minor versions — for example, when using the 8.5 syntax mode and linking PHP 8.4, whether the final build succeeds is still determined by the Zend APIs actually used; |
||||
- environments below PHP 8.4 are rejected directly at the TypePHP/PHPX build entry point. |
||||
|
||||
## 6. TDD coverage |
||||
|
||||
Add failing tests before implementation, covering: |
||||
|
||||
1. get-only, set-only, and get/set Interface contracts; |
||||
2. ordinary backed properties, Hooked Properties, and inherited properties satisfying the contract; |
||||
3. compile errors for missing properties, missing get/set, non-public, and incompatible types; |
||||
4. Interface inheritance, merging of multiple contracts, and conflicts; |
||||
5. Reflection abstract, virtual, hasHook/getHook metadata; |
||||
6. success and failure linking of PHP 8.4 dynamic PHP classes; |
||||
7. consistent O0/O3 results, with Interfaces generating no property slots or Native Hook implementations; |
||||
8. lifetime and ABI regression of the PHPX helper under NTS/ZTS and PHP 8.4/8.5. |
||||
|
||||
## 7. Implementation order |
||||
|
||||
1. Add TP-AOT-010 normal-scenario and syntax-error PHPT and confirm the current failures. |
||||
2. Add the Interface Property Contract model and preprocessing collection logic. |
||||
3. Implement Interface inheritance and compile-time contract checking for implementing classes. |
||||
4. Add the abstract Hook metadata helper in PHPX. |
||||
5. Modify stub generation and class registration order to wire into the PHP 8.4 Zend metadata. |
||||
6. Add Reflection, dynamic class linking, target version, and generated-code tests. |
||||
7. Run the Interface, Property Hook, Reflection, and full compiler regression. |
||||
|
||||
After completion, runtime property access still goes directly into the implementing class's ordinary properties or Native Hooks; the Interface contract itself exists only in the compile-time model and MINIT metadata, and does not enter the request hot path. |
||||
@ -0,0 +1,67 @@ |
||||
# Automatically building libphp.so |
||||
|
||||
TypePHP's executable and shared-library modes require the `libphp.so` provided by the PHP Embed SAPI. Many Linux distributions' PHP packages only include CLI or FPM, so the Composer-installed `tpc.php` asks whether to automatically build a private PHP when `libphp.so` is not found. |
||||
|
||||
The installer targets `tpc.php` launched by the PHP interpreter. The bootstrap binary `tpc` must have `libphp.so` and `libphpx.so` loaded by the system dynamic linker before entering `main()`, so it cannot install missing libraries by itself. |
||||
|
||||
This feature is only enabled on Linux and in interactive terminals. Extension mode (`-m ext`) does not need `libphp.so` and does not trigger the installer; non-interactive environments such as CI also do not automatically download or install packages, or execute `sudo`. |
||||
|
||||
## Usage flow |
||||
|
||||
Simply run the normal compile command: |
||||
|
||||
```bash |
||||
vendor/bin/tpc.php project.yml |
||||
``` |
||||
|
||||
When `libphp.so` is missing, the installer asks in sequence: |
||||
|
||||
1. whether to automatically build the PHP Embed library; |
||||
2. the PHP version, defaulting to exactly the `PHP_VERSION` of the currently running `tpc.php`, with the option to manually enter another PHP 8.4.x/8.5.x stable version; |
||||
3. the install directory, defaulting to `~/.typephp`; |
||||
4. whether to install missing development packages via the detected `apt-get`, `dnf`, or `yum`. |
||||
|
||||
The installer reads the current `php-config --configure-options`, keeps the current PHP's extension configuration, replaces the install path, and adds `--enable-embed=shared`. PHP source is downloaded only from PHP.net, and verified using the SHA-256 from the official release information. |
||||
|
||||
After compilation, the main files are as follows: |
||||
|
||||
```text |
||||
~/.typephp/bin/php |
||||
~/.typephp/bin/php-config |
||||
~/.typephp/lib/libphp.so |
||||
~/.typephp/lib/php.ini |
||||
~/.typephp/lib/loaded-extensions.txt |
||||
``` |
||||
|
||||
The current main ini file and the configuration in the scan directory are merged. When the same PHP major/minor version is used, the shared extensions loaded in the current ini are copied to the new extension directory; across major/minor versions, binary extensions are not copied, and unusable extension configuration is commented out to avoid a generated PHP that cannot start. |
||||
|
||||
After a successful installation, the current `tpc.php` process automatically uses the new directory as `PHP_HOME` and continues the original compile task. It can also be specified explicitly later: |
||||
|
||||
```bash |
||||
export PHP_HOME="$HOME/.typephp" |
||||
vendor/bin/tpc.php project.yml |
||||
``` |
||||
|
||||
When the same directory and version are chosen again, the installer asks whether to directly reuse the existing `libphp.so` and does not repeat the full build. |
||||
|
||||
## Non-interactive environments |
||||
|
||||
The installer does not automatically confirm privileged operations in CI. Prepare `libphp.so` in advance, then set: |
||||
|
||||
```bash |
||||
PHP_HOME=/path/to/php vendor/bin/tpc.php project.yml |
||||
``` |
||||
|
||||
## Automatically building libphpx.so |
||||
|
||||
After the PHP Embed check completes on Linux, the build pipeline also checks `lib/libphpx.so` in the PHPX root directory. The PHPX root directory is resolved in the following order: |
||||
|
||||
1. `PHPX_HOME`; |
||||
2. the `swoole/phpx` install path in Composer `InstalledVersions`; |
||||
3. `vendor/swoole/phpx` within the TypePHP source repository. |
||||
|
||||
When the shared library is missing, an interactive terminal asks whether to build. PHPX itself does not depend on `libphp.so`; it depends on PHP headers and `php-config`. `LibPhpxInstaller` uses the currently selected PHP prefix, sets `PHP_HOME`, puts that prefix's `bin` at the front of `PATH`, and passes `-Dphp_dir=<PHP prefix>` to the PHPX CMake, ensuring PHPX matches the PHP ABI used by the project runtime. |
||||
|
||||
Toolchain detection runs after the local library checks. After the user confirms the automatic build, the installer checks and can install GCC/G++, make, CMake, pkg-config, and the dependencies required by PHP configure via `apt-get`, `dnf`, or `yum`; therefore a Composer environment does not need a full pre-installed C/C++ toolchain. |
||||
|
||||
The build always uses Release, disables PHPX tests, and builds only the `phpx` target; parallelism is capped at 8. The output must be `<PHPX>/lib/libphpx.so`, otherwise the installer reports an error. Non-interactive environments only report the missing library and do not run CMake. |
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,136 @@ |
||||
# Native Class Implementation Acceptance Matrix |
||||
|
||||
> Audit date: 2026-08-17 |
||||
> This document records the requirements, implementation entry points, and direct verification evidence for the `#[Native]` object model. It is the implementation-acceptance attachment for |
||||
> [NATIVE_CLASS_OBJECT.md](NATIVE_CLASS_OBJECT.md) and does not replace the semantic design document. |
||||
|
||||
## 1. Acceptance principles |
||||
|
||||
Every capability must simultaneously have: |
||||
|
||||
1. a clear language boundary; |
||||
2. a locatable compiler or PHPX implementation; |
||||
3. direct evidence in positive PHPT, negative PHPUnit, or PHPX C++ unit tests. |
||||
|
||||
Code alone, documentation alone, or "no failure currently observed" does not count as complete. Native Objects have no Zend representation, so any cross-boundary behavior that cannot be statically proven safe must be rejected before generating C++. |
||||
|
||||
## 2. Object model and code generation |
||||
|
||||
| Requirement | Implementation evidence | Test evidence | Conclusion | |
||||
|---|---|---|---| |
||||
| `#[Native]` used only on named classes | `NativeClassAttributeLowering`, `NativeClassSupportTrait` | `testRejectsNativeAttributeOnInterface/Trait/Enum/AnonymousClass` | Verified | |
||||
| No Zend class/object handlers registered | Native struct, descriptor, and free-function generation path | `clone-and-zend-invisible.phpt`, Reflection negative tests | Verified | |
||||
| Methods keep the `php_*` free-function ABI | Native method/virtual thunk generation path | `basic.phpt`, `chained-call.phpt` | Verified | |
||||
| Statically resolvable `new NativeClass()` uses the Native Heap | `CompilerBase::parseNew()`, `php::nativeConstruct()` | `basic.phpt`, `construction-gc-roots.phpt` | Verified | |
||||
| `new (expression)()` stays ordinary PHP dynamic instantiation | `parseNew()` enters the Native branch only for `Node\\Name` | `testLeavesDynamicClassExpressionsToTheOrdinaryPhpPath` | Verified | |
||||
| A Native object itself cannot serve as a dynamic class target | `assertNotNativeObjectDynamicClassTarget()` | dynamic new/static call/class constant negative tests | Verified | |
||||
| All unsupported usages terminate at compile time | Native boundary checks, type compatibility checks | 131 `NativeClassValidationTest` items | Verified | |
||||
|
||||
## 3. Properties and fixed layout |
||||
|
||||
| Requirement | Implementation evidence | Test evidence | Conclusion | |
||||
|---|---|---|---| |
||||
| All properties must declare types | Native field validation | `testRejectsUntypedProperty` | Verified | |
||||
| bool/int/float use fixed value fields | Native field C++ type mapping | `basic.phpt`, `numeric-properties.phpt` | Verified | |
||||
| string/array/object/typed object/Stream/mixed usable as fields | Native PHPX field mapping, write checks | `phpx-properties.phpt`, `stream-property.phpt`, `composite-property-types.phpt` | Verified | |
||||
| BigInt/BigFloat/Decimal usable as fields | high-precision field mapping and trace/destroy | `high-precision-properties.phpt` | Verified | |
||||
| Native type fields hold raw pointers and can form cyclic types | struct forward declaration, descriptor trace | `mutual-reference-types.phpt`, `gc-cycle.phpt` | Verified | |
||||
| Fields without explicit initialization use deterministic zero values | Native field initializer | `zero-values.phpt` | Verified | |
||||
| Property writes keep the declared type | Native property assignment validation | composite, stream, and multiple negative PHPUnit | Verified | |
||||
| Only `any` properties may take PHP references | Native property reference lowering | `any-property-reference.phpt` and mixed/fixed property negative tests | Verified | |
||||
| `unset()` not supported on Native properties | property unset validator | `testRejectsUnsetOnNativeObjectProperties` | Verified | |
||||
| readonly properties not supported | Native declaration validator | `testRejectsReadonlyPropertyUntilNativeWriteStateIsImplemented` | Verified | |
||||
| Box/Std Container cannot be embedded in fields | Native field validator | Box/Std Container property negative tests | Verified | |
||||
|
||||
## 4. Identity, nullability, and call ABI |
||||
|
||||
| Requirement | Implementation evidence | Test evidence | Conclusion | |
||||
|---|---|---|---| |
||||
| `$a = $b` only copies the pointer and shares object identity | Native pointer local representation | `parameter-semantics.phpt` | Verified | |
||||
| Native parameters and returns must explicitly declare concrete classes | call argument/return boundary validation | untyped/mixed/interface parameter and return negative tests | Verified | |
||||
| Ordinary Native parameters are non-null; only `?Class` may be null | function entry/return checks | `non-null-parameter.phpt`, `nullable-signatures.phpt`, `return-nullability.phpt` | Verified | |
||||
| `&` forbidden on Native parameters, returns, and variables | reference boundary validation | reference parameter/return/assignment/function/method negative tests | Verified | |
||||
| Native variadic, union/intersection signatures not supported | signature validation | variadic/union/null-union negative tests | Verified | |
||||
| `unset($object)`/`$object = null` only clear the current pointer slot | Native root slot lowering | `unset-alias.phpt` | Verified | |
||||
| `===`/`!==` and `match` use pointer identity | Native identity lowering | `strict-identity.phpt`, `match-identity.phpt` | Verified | |
||||
| ternary/match/coalesce choose the nearest common Native base class for sibling subclasses | `getCommonNativeObjectClass()`, selection pointer cast | `value-selection.phpt`, cross-file global discovery tests | Verified | |
||||
| Conditional expressions check for non-null pointer without calling `toBool()` | Native condition lowering | `conditions.phpt` | Verified | |
||||
| Loose comparison, arithmetic, bitwise, increment/decrement, compound writes, and switch forbidden | operator validators | corresponding PHPUnit negative tests | Verified | |
||||
| `isset`/`empty`/`is_null`/nullsafe keep the typed pointer | Native selection/nullsafe lowering | `isset-empty.phpt`, `is-null.phpt`, `nullsafe.phpt` | Verified | |
||||
| Call arguments strictly evaluated left-to-right and precisely rooted at safe points | Native call argument materialization | `call-argument-roots.phpt`, `constructor-argument-roots.phpt` | Verified | |
||||
|
||||
## 5. Class language capabilities |
||||
|
||||
| Requirement | Implementation evidence | Test evidence | Conclusion | |
||||
|---|---|---|---| |
||||
| Single inheritance, abstract, and limited virtual dispatch | Native C++ inheritance/virtual slot adapters | `abstract-method.phpt`, `polymorphic-clone.phpt`, `virtual-signature-variance.phpt` | Verified | |
||||
| public/private/protected checked at compile time | Native member resolution | `method-visibility.phpt` and inaccessible method/constant negative tests | Verified | |
||||
| Traits compiled as ordinary Native members after injection | existing Trait AST injection + Native member generation | `trait-inheritance-interface.phpt` | Verified | |
||||
| Interfaces are compile-time contracts only and cannot become value representations | interface contract validator | `internal-interface.phpt`, `interface-property-hooks.phpt`, and interface escape negative tests | Verified | |
||||
| Compile-time resolvable `instanceof` folds | Native instanceof lowering | `instanceof.phpt`, dynamic instanceof negative tests | Verified | |
||||
| Getter/Setter annotations generate direct calls | annotation lowering + Native method path | `generators.phpt` | Verified | |
||||
| Property Hooks support only direct get/set | Native hook lowering | `property-hooks.phpt`, `property-hook-native-object.phpt`, and indirect operation negative tests | Verified | |
||||
| `clone` preserves dynamic subclass, PHPX COW, and shallow object semantics | Native clone descriptor/thunk, `php::nativeClone()` | clone PHPT series, `clone-phpx-fields.phpt` | Verified | |
||||
| `__construct` called only by `new` | Native construction path, explicit-call checks | construction PHPT series, explicit constructor negative tests | Verified | |
||||
| `__destruct` executed at most once by GC, derived-to-base along the inheritance chain | Native finalizer chain | destructor/finalizer/lifecycle PHPT series | Verified | |
||||
| `__invoke` and `__toString` use a deterministic Native Call | Native magic method allow-list | `magic-methods.phpt` | Verified | |
||||
| Dynamic magic methods, variable property/method names not supported | Native magic/dynamic access deny-list | dynamic magic, variable method/property negative tests | Verified | |
||||
| `toArray/toString/toInt/toFloat/toBool/toObject` require a real method, zero parameters, and an exact return type | Native keyword method resolution | `keyword-conversions.phpt`, `testNativeObjectToObjectKeywordUsesDeclaredNativeMethod`, and signature negative tests | Verified | |
||||
| `count($obj)` specialized only when implementing Countable | Native count optimizer | `keyword-conversions.phpt`, count-without-countable negative tests | Verified | |
||||
| `ArrayAccess` direct syntax maps to Native `offset*()` methods | Native array access lowering | `array-access.phpt` | Verified | |
||||
| Native `ArrayAccess` forbids indirect modification and references | writable-chain/reference validators | ArrayAccess compound/increment/nested/property/reference/coalesce negative tests | Verified | |
||||
| Native `Iterator` foreach maps to protocol methods, preserving PHP call order | Native foreach lowering | `iterator.phpt` | Verified | |
||||
| `IteratorAggregate` routes Native Iterator vs PHP Traversable | aggregate return-type lowering | `iterator.phpt` | Verified | |
||||
| Native foreach does not enumerate properties and forbids by-reference traversal | interface/reference validators | foreach negative PHPUnit | Verified | |
||||
|
||||
## 6. GC and lifetime |
||||
|
||||
| Requirement | Implementation evidence | Test evidence | Conclusion | |
||||
|---|---|---|---| |
||||
| Wren-style precise, non-moving, STW mark-sweep | `phpx/thirdparty/wren-gc`, `native_gc.cc` | PHPX `wren_gc.*` | Verified | |
||||
| Raw pointer writes have no RC, no write barrier | Native pointer field/local codegen | generated C++ review, Native PHPT | Verified | |
||||
| 16 MiB initial threshold, 1 MiB lower bound, 50% headroom | Wren GC configuration | `wren_gc.uses_stable_native_heap_defaults` | Verified | |
||||
| Precise root frames keep the object graph alive | `NativeRootFrame`, generated root slots | PHPX root tests, `gc-cycle.phpt` | Verified | |
||||
| Fiber non-LIFO lifetime safety | root frame registry | `fiber-lifetime.phpt`, `fiber-shutdown.phpt`, PHPX Fiber root tests | Verified | |
||||
| global/static request roots are thread-local under ZTS | generated globals/root registration | `global-and-static.phpt` under ZTS, PHPX request root tests | Verified | |
||||
| RSHUTDOWN clears roots and destroys the heap | `nativeGcRequestShutdown()` | PHPX shutdown tests | Verified | |
||||
| A finalizer can resurrect once and is not re-executed afterward | Wren/Native finalization state | `gc-cycle.phpt`, PHPX resurrection tests | Verified | |
||||
| Allocation, exceptions, and Zend state safe in finalizers | finalizer queue/exception cleanup | finalizer/lifecycle PHPT, PHPX finalizer tests | Verified | |
||||
| Construction or clone failure leaves no dangling object, and escaped objects remain valid | `nativeConstruct()`, `nativeClone()` failure paths | `failed-lifecycle-escape.phpt`, `failed-clone-finalizer.phpt` | Verified | |
||||
|
||||
## 7. ZendVM boundary and containers |
||||
|
||||
| Requirement | Implementation evidence | Test evidence | Conclusion | |
||||
|---|---|---|---| |
||||
| Native Objects cannot enter PHP array/object property/mixed | escape and boundary validators | corresponding PHPUnit negative tests | Verified | |
||||
| Cannot be passed to PHP/ZendVM dynamic functions, Closures, or constructors | call boundary validator | dynamic call, Closure, Zend constructor negative tests | Verified | |
||||
| Reflection/WeakReference/serialize/json_encode not supported | facility-specific diagnostics | corresponding PHPUnit negative tests | Verified | |
||||
| Generators cannot hold, receive, or yield Native pointers | generator boundary validator | generator series negative tests | Verified | |
||||
| Native locals of ordinary functions across Fiber suspend have precise roots | root frame lifecycle | Fiber PHPT | Verified | |
||||
| Local Std Containers can hold concrete Native pointers | Std Container Native value mapping/root frame | `std-containers.phpt` | Verified | |
||||
| Native Std Containers cannot escape as Zend values, static/global, or closure capture | container escape validation | Std Container series negative PHPUnit | Verified | |
||||
| `include`/`eval` do not expose Native locals to the Zend symbol table | include scope filtering | `include-native-scope.phpt` | Verified | |
||||
|
||||
## 8. Project-level analysis |
||||
|
||||
| Requirement | Implementation evidence | Test evidence | Conclusion | |
||||
|---|---|---|---| |
||||
| Native class forward declarations do not depend on file order | declaration discovery pre-pass | `testDiscoversNativeTypesBeforeCrossFileSignaturePreprocessing` | Verified | |
||||
| Global Native slot ABI fixed before any C++ file is generated | `NativeGlobalDiscovery`, `NativeGlobalTypeResolver` | `testDiscoversNativeGlobalSlotBeforeEarlierReaderIsConverted`, actual dual-file build | Verified | |
||||
| `global $slot` and statically resolvable `$GLOBALS[...]` use the same Native root slot | literal/constant global slot lowering, request root registration | `global-and-static.phpt`, cross-file Closure/constant `$GLOBALS` fixture | Verified | |
||||
| Dynamic `$GLOBALS[$key]` must not carry Native Objects | dynamic Zend boundary validation | `testRejectsNativeObjectStoredThroughDynamicGlobalsKey` | Verified | |
||||
| Global slot fixes the first Native type and allows only subclasses or null | global registration/type validation | `global-and-static.phpt`, global type change negative tests | Verified | |
||||
| Projects without Native Classes skip the Native global pre-pass | `discoverNativeGlobalObjects()` fast return | source review, full PHPUnit | Verified | |
||||
|
||||
## 9. Current verification commands |
||||
|
||||
```bash |
||||
./run-tests.php -j4 --compiler ./tpc tests/compiler/native-class/ |
||||
vendor/bin/phpunit phpunit/src/NativeClass/NativeClassValidationTest.php |
||||
/home/swoole/workspace/aot/phpx/build/bin/phpx-tests \ |
||||
--gtest_filter='wren_gc.*:native_gc.*' |
||||
``` |
||||
|
||||
The results of this Iterator-focused run were: `iterator.phpt` 1/1, Native Class PHPUnit 136/136, |
||||
ordinary foreach regression 14/14. The Native Class PHPT directory currently has 71 items; per the current task agreement, the full |
||||
tests for that directory and the compiler PHPT suite were not re-run this time, and are left for the next unified regression round. |
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,212 @@ |
||||
# Zend Object Creation and Property Default Value Initialization |
||||
|
||||
This document records the initialization responsibilities of the Zend Classes generated by TypePHP during MINIT and object creation, focusing on when a custom `create_object` is required, what behavior is allowed within it, and the performance boundary on the object-creation hot path. |
||||
|
||||
This document only discusses ordinary TypePHP Classes registered with ZendVM. `#[Native]` Classes use the Native Heap and GC and do not follow the flow described here. |
||||
|
||||
## 1. The two initialization stages must be kept separate |
||||
|
||||
Property initialization of a TypePHP Class is split into two stages: |
||||
|
||||
1. `gen_stub.php` generates `register_class_*()` during MINIT, establishing the `zend_class_entry`, property metadata, and the default property table; |
||||
2. only values that cannot be accurately expressed by the default property table are supplemented by a custom `create_object` each time an object is created. |
||||
|
||||
These two stages must not perform the same property assignment twice. Values already written by `register_class_*()` are copied to the new object by Zend's `object_properties_init()`; calling `zend_update_property()` again has no semantic value and additionally enters the property-name lookup, type check, handler dispatch, and reference-counting paths. |
||||
|
||||
## 2. Default values handled by gen_stub.php |
||||
|
||||
The following values can be accurately written into the Zend Class default property table: |
||||
|
||||
| Source default value | Registration-stage representation | Must be written again in `create_object` | |
||||
|---|---|---| |
||||
| `null` | `ZVAL_NULL` | No | |
||||
| `bool` | `ZVAL_TRUE/FALSE` | No | |
||||
| `int` | `ZVAL_LONG` | No | |
||||
| `float` | `ZVAL_DOUBLE` | No | |
||||
| `string` | persistent `zend_string` | No | |
||||
| scalar constant expression | the scalar zval evaluated at compile time | No | |
||||
| `[]` | `ZVAL_EMPTY_ARRAY` | No | |
||||
| TypePHP typed property without an explicit default | the zero value, empty string, empty array, `null`, or `UNDEF` defined by TypePHP | No | |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
class Value |
||||
{ |
||||
private const BASE = 20; |
||||
|
||||
public int $id = self::BASE + 3; |
||||
public string $name = 'type' . 'php'; |
||||
public array $items = []; |
||||
} |
||||
``` |
||||
|
||||
As long as the expressions can be safely evaluated at compile time, the three properties above should rely entirely on the Zend Class default property table. When creating `Value`, `zend_update_property()` must not be called again. |
||||
|
||||
## 3. When default values need runtime supplementation |
||||
|
||||
The current `gen_stub.php` cannot accurately represent the following values in the default property table. |
||||
|
||||
### 3.1 Non-empty arrays |
||||
|
||||
Non-empty array defaults currently use `ZVAL_EMPTY_ARRAY` as a placeholder value in the registration function. Each object must construct an independent, semantically correct array value: |
||||
|
||||
```php |
||||
class Request |
||||
{ |
||||
public array $options = ['timeout' => 10]; |
||||
} |
||||
``` |
||||
|
||||
Therefore `Request::$options` needs to be supplemented in `create_object`. Multiple objects still follow PHP array copy-on-write semantics; modifying one object's array must not affect other objects. |
||||
|
||||
Array constants follow the same rule. If the compiler can only determine that it is an array but cannot prove it is empty, it conservatively keeps the runtime initialization. |
||||
|
||||
### 3.2 Enum case |
||||
|
||||
An enum case is an object, not a scalar constant: |
||||
|
||||
```php |
||||
enum State |
||||
{ |
||||
case Ready; |
||||
} |
||||
|
||||
class Task |
||||
{ |
||||
public State $state = State::Ready; |
||||
} |
||||
``` |
||||
|
||||
The class registration code currently can only generate a placeholder value first; `create_object` then obtains the real enum case object and writes it into the property. Therefore "only non-empty arrays need a custom `create_object`" is not correct — enum case is a clear second category of counterexample. |
||||
|
||||
### 3.3 Constant expressions that cannot be safely resolved |
||||
|
||||
If the preprocessing stage cannot prove that a default value can be accurately expressed by the Zend default property table, the compiler must conservatively keep the runtime initialization. Optimization can only remove work that is proven redundant; it must not guess the runtime type based on the expression's shape. |
||||
|
||||
## 4. handlers and parent allocator |
||||
|
||||
### 4.1 Property Hooks and asymmetric set visibility do not trigger on their own |
||||
|
||||
PHP 8.4 Property Hooks, `private(set)`, and `protected(set)` install TypePHP custom object handlers, but this by itself does not require overriding `create_object`. Zend 8.4's `object_properties_init()` directly copies the class default table and does not call read/write handlers; ordinary `php::stdCreateObject()` already sets the final handlers correctly. |
||||
|
||||
A custom creation flow is required only when the class also has runtime defaults such as non-empty arrays or enum cases. Supplemental initialization must bypass setters; even using `zend_std_write_property()`, PHP 8.4 would call the setter based on the Hook metadata. The current generated code therefore uses the property offset known at compile time to directly update the backing slot via PHPX `Object::attr(offset)`. |
||||
|
||||
### 4.2 Parent custom object allocator |
||||
|
||||
If the parent class comes from a built-in PHP extension, or an ancestor class has a custom object storage layout, the child class cannot bypass the parent's allocator. When the current class genuinely needs a custom creation flow due to runtime defaults, it must first call the saved parent `create_object`, then supplement the current class's values. |
||||
|
||||
When the TypePHP parent has already installed a custom allocator, ordinary child classes usually inherit it directly. A new delegation layer is generated only when the child class itself also needs supplemental initialization. |
||||
|
||||
## 5. Execution flow of a custom create_object |
||||
|
||||
The generated code performs the following steps through `typephp_create_object_with_defaults()`: |
||||
|
||||
1. save the class's final `default_object_handlers`; |
||||
2. if the parent object layout must be respected, call the saved parent allocator; otherwise run `zend_objects_new()` and `object_properties_init()`; |
||||
3. temporarily switch the new object to Zend standard object handlers to keep the exception path and other object operations in a controlled state; |
||||
4. initialize only the properties marked `requiresRuntimeDefaultInit`, writing directly to backing slots via cached declared-property offsets; |
||||
5. check for Zend exceptions after each write; |
||||
6. restore the final handlers whether returning normally or after a C++ exception; |
||||
7. return the fully initialized `zend_object *`. |
||||
|
||||
The initializer is a template parameter and a compile-time lambda; it does not use `std::function` and does not dynamically allocate memory for the lambda. `delegate_to_base` is a call-site-determined boolean that can usually be folded by the C++ compiler in optimized builds. |
||||
|
||||
The following behavior does not belong to `create_object`: |
||||
|
||||
- the function body of PHP `__construct()`; |
||||
- static property default-value initialization; that happens in `module_init()`; |
||||
- scalar, `null`, and empty-array assignments already expressed by the default property table; |
||||
- reapplying defaults after clone; clone should copy the source object's current state, not recreate the default state. |
||||
|
||||
## 6. Main performance issues already fixed |
||||
|
||||
The old generation logic installed a custom `create_object` whenever any explicit non-static default existed in the class, and re-updated all default properties on every object creation. This produced two layers of duplicate cost: |
||||
|
||||
1. an ordinary class containing only `public int $value = 0` also bypassed the standard fast creation path; |
||||
2. a class containing even one non-empty array caused all other scalar properties to be re-updated one by one. |
||||
|
||||
The current rules have been adjusted to: |
||||
|
||||
- only properties that truly need runtime supplementation trigger `requireCtor`; |
||||
- properties already accurately registered by `gen_stub.php` do not appear in the runtime initialization block; |
||||
- classes with only Hooks/asymmetric visibility and no runtime defaults no longer generate an empty custom allocator; |
||||
- when Hooks and runtime defaults coexist, a fixed property offset updates the backing slot without calling the setter. |
||||
|
||||
In a micro benchmark, `new Foo()` containing only scalar properties dropped from about `1.8s` to about `0.78s`, close to the approximately `0.83s` of ZendPHP in the same environment after subtracting the empty loop. This number is only used to record the magnitude of the optimization, not a cross-machine performance promise. |
||||
|
||||
## 7. Implemented optimizations, remaining costs, and future directions |
||||
|
||||
### 7.1 Non-empty arrays use a request-level template and copy-on-write |
||||
|
||||
Non-empty arrays cannot be placed in an internal class's default property table, but that does not mean the array must be rebuilt for every object. The current generator already uses request-level default-value templates: |
||||
|
||||
1. each class containing runtime array defaults owns a set of `THREAD_LOCAL php::Var` templates and an initialization state; NTS builds introduce no locking; |
||||
2. the first time an object of the class is created, all its templates are lazily built via `UNEXPECTED(!initialized)`; |
||||
3. templates are committed and the initialization flag is set only after all templates are successfully built in local temporaries; construction exceptions do not publish a half-initialized state; |
||||
4. template initialization happens before object allocation, so a failure leaves no unreturned object; |
||||
5. subsequent object creations just copy the template zval into the target backing slot, i.e. increment the array reference count once; |
||||
6. the first time an object modifies that property, Zend/PHPX's `SEPARATE_ARRAY` performs copy-on-write; |
||||
7. `module_clean()` releases the templates and resets the initialization state; the HashTable allocated by the request allocator does not survive RSHUTDOWN. |
||||
|
||||
Take the following default value as an example: |
||||
|
||||
```php |
||||
class Request |
||||
{ |
||||
public array $options = [ |
||||
'timeout' => 10, |
||||
'headers' => ['Accept' => 'application/json'], |
||||
]; |
||||
} |
||||
``` |
||||
|
||||
If ten thousand objects are created but `$options` is not modified, the array and nested arrays are built only once; each object only holds the shared zval. If one object executes `$request->options['timeout'] = 30`, only that object is separated at write time, while the other objects and the template remain unchanged. Nested arrays also continue to use Zend's existing per-level copy-on-write rules. |
||||
|
||||
PHP property default arrays cannot contain references, and the objects allowed in constant expressions are mainly immutable enum cases, so sharing the template conforms to default-property semantics. PHPT already covers top-level writes, nested writes, `unset`, reference writes, and dynamic object writes, confirming that these paths all separate correctly. |
||||
|
||||
A persistent array cannot simply be constructed in MINIT and passed to `zend_declare_typed_property()`. TypePHP registers `ZEND_INTERNAL_CLASS`, and Zend 8.4 explicitly forbids internal properties from using refcounted default zvals; the internal-class fast path of `_object_properties_init()` also does not increment the reference count of defaults. Non-empty arrays and enum objects are both refcounted values. |
||||
|
||||
Therefore, without changing the foundational design of "TypePHP Classes are registered as internal classes" and without modifying the Zend ABI, non-empty arrays still do not enter the class default table; the out-of-table request-level template reduces array construction cost from "once per object" to "once per request per default value". Objects that do not modify the default array only bear the zval copy and reference-counting cost; only objects that actually modify it bear the array-separation cost. |
||||
|
||||
Templates are initialized lazily per class rather than unconditionally building all templates in RINIT: in large projects, many classes are never instantiated within a single request. Each object only adds one highly predictable initialization-state branch; after the first time, the branch stably evaluates to false. |
||||
|
||||
A persistent immutable template with module lifetime is not generated for now. That approach requires fully validating persistent HashTables, interned strings, nested arrays, MSHUTDOWN, and ZTS, and arrays containing runtime constants or enum cases would still need the request-level path. Until ZendVM's constraints on these combinations are sufficiently validated, the request-level template is the safety boundary. |
||||
|
||||
### 7.2 Changed to fixed property-slot writes |
||||
|
||||
Properties supplemented at runtime already have their class, property name, offset, and type known at compile time. The current implementation reuses the persistent property-offset cache and updates slots via `php::Object::attr(offset)`, already eliminating the property-name hash lookup, the generic write handler, and the Property Hook setter on every object. |
||||
|
||||
It still builds a short-lived `php::Object` carrier for the initializer and reads the offset cache. If profiling later proves this is a hot spot, the final offset can be saved directly after MINIT, or PHPX can add an initialization helper that does not take object ownership. Any further optimization must continue to handle old-value destruction, reference counting, parent-class private slots, Hook backing slots, and exception safety, and must not regress to unprotected raw-pointer assignment. |
||||
|
||||
### 7.3 Enum case can be bound early |
||||
|
||||
An enum case is likewise a refcounted object and cannot directly serve as an internal-class default zval. One could cache the stable enum case pointer or zval in MINIT and then perform correct reference-count copying on each object creation, eliminating the repeated class/case lookup; the object property write itself still cannot be omitted. |
||||
|
||||
### 7.4 Multi-layer allocators on the inheritance chain |
||||
|
||||
When both parent and child classes have runtime defaults, the creation flow delegates layer by layer and runs each initialization, with cost growing with the number of involved inheritance layers. In the future, inheritance chains fully controlled by TypePHP with no special object layout could have their initialization plans merged; built-in-extension parent classes must still call their allocator. |
||||
|
||||
### 7.5 Conservative constants can produce unnecessary allocators |
||||
|
||||
Constants that cannot be resolved in the preprocessing stage conservatively enter the runtime path. A unified constant-default classification pass could be added after symbol preparation to reduce custom allocators for cases that are "actually scalars but unprovable early". That optimization must preserve the distinction between enum cases and array constants. |
||||
|
||||
### 7.6 Dynamic access cost of custom handlers |
||||
|
||||
TypePHP currently installs property handlers for ordinary Zend Classes to support typed-property unset semantics, Property Hooks, and asymmetric write visibility. Installation happens in MINIT and is not equivalent to installing a custom `create_object`; however, dynamic property reads/writes may still enter the handler. Native property accesses already resolved to fixed slots by the compiler must not degrade because of this. |
||||
|
||||
## 8. Regression test requirements |
||||
|
||||
Changes to this flow should at least cover: |
||||
|
||||
- scalars, scalar constant expressions, and empty arrays do not generate a custom allocator; |
||||
- non-empty arrays generate an allocator, and array modifications on two objects do not affect each other; |
||||
- enum case defaults are real enum objects after object creation; |
||||
- classes containing only Property Hooks or asymmetric set visibility do not generate an empty allocator, and Reflection and dynamic read/write behavior do not degrade; |
||||
- Property Hook/asymmetric properties combined with runtime defaults do not trigger setters; |
||||
- when parent and child classes each declare runtime defaults, both parent and child properties are correct; |
||||
- inheriting a built-in extension class does not break its object layout; |
||||
- the exception path restores object handlers; |
||||
- bootstrap compilation and full PHPUnit/PHPT regression pass. |
||||
|
||||
Current core assertions on code generation live in `NewObjectCodegenTest`; runtime semantics are covered by `default-initialization-paths.phpt`, `default-expressions-inheritance.phpt`, and the Property Hook test group. |
||||
@ -0,0 +1,301 @@ |
||||
# TypePHP's Three Object Storage and Passing Models |
||||
|
||||
> Status: current architectural constraint. This document explains why TypePHP simultaneously keeps three object-style value models — Zend Object, PHPX Box, and |
||||
> Native Class Object — along with their respective ownership, passing methods, and boundaries. |
||||
|
||||
## 1. Conclusion |
||||
|
||||
TypePHP currently has three object storage and passing mechanisms: |
||||
|
||||
1. Ordinary PHP/Zend Objects; |
||||
2. PHPX Box, including Std Containers and high-precision types; |
||||
3. `#[Native]` Native Class Objects. |
||||
|
||||
These three are not historical residue of the same design, but separately solve three mutually conflicting problems: |
||||
|
||||
- Zend Object preserves PHP's dynamic object semantics and ZendVM ecosystem compatibility; |
||||
- Box provides an opaque Zend value carrier for C++ types that cannot be fully written into PHP type declarations; |
||||
- Native Class Object provides statically-knowable business objects with a fixed layout close to C/C++, raw-pointer calls, and |
||||
tracing GC. |
||||
|
||||
None of these mechanisms can replace the other two without losing a core capability of the others. The current design explicitly accepts |
||||
the long-term coexistence of the three models and does not target a "unified object representation". |
||||
|
||||
## 2. Overview |
||||
|
||||
| Dimension | Zend Object | PHPX Box | Native Class Object | |
||||
| --- | --- | --- | --- | |
||||
| Typical value | Ordinary PHP class instance | Std Container, BigInt, BigFloat, Decimal | `#[Native] class` instance | |
||||
| Primary representation | `zend_object` / zval | `zend_resource` + `php::Box *` | C++ struct in Native Heap + raw pointer | |
||||
| Type identity | `zend_class_entry *` | Box C++ dynamic type, `type_info`/type ID | Compile-time Native class, dynamic type saved in descriptor | |
||||
| Lifecycle | Zend reference counting + Zend cycle GC | Zend resource reference counting calling the Box destructor | Wren-style precise, non-moving mark-sweep GC | |
||||
| Argument passing | `php::Object` / `php::Var`, copying the handle and adjusting RC | `php::Var` carrying the resource; hot paths extract the concrete C++ reference | Concrete `NativeClass *` passed by value, without adjusting RC | |
||||
| Property/method access | Zend handlers, dynamic lookup, or already-cached Native Call | The compiler generates operations based on the concrete Box type | Fixed-offset field access and definite `php_*` Native Call | |
||||
| Dynamic PHP interop | Complete | Limited interop as an opaque resource | Cannot enter the ZendVM value boundary | |
||||
| Cyclic graph handling | Zend GC can scan the Zend object graph | Zend GC does not scan the C++ object graph inside Box | The Native descriptor precisely traces the Native pointer graph | |
||||
| Core goal | PHP compatibility | Carrying C++ generic/extended values | Extreme static performance | |
||||
|
||||
## 3. Ordinary PHP/Zend Object |
||||
|
||||
### 3.1 Storage |
||||
|
||||
Ordinary classes are registered with the ZendVM, and instances are represented by `zend_object`. TypePHP holds the corresponding zval through PHPX RAII types such as `php::Object`, |
||||
`php::Variant`/`php::Var`. |
||||
|
||||
The object has Zend's class entry, property table, object handlers, and method metadata. Based on compile-time information, |
||||
TypePHP can optimize some accesses into definite Native Calls, but the object identity and lifecycle still belong to the ZendVM. |
||||
|
||||
### 3.2 Passing and Lifecycle |
||||
|
||||
PHP object assignment and argument passing copy the object handle, not the object entity, and follow Zend reference counting. Cyclic references in the object graph |
||||
are handled by Zend GC. Objects can naturally enter: |
||||
|
||||
- PHP arrays and ordinary object properties; |
||||
- `mixed`/`object` variables; |
||||
- Closures, Generators, Fibers, and dynamic calls; |
||||
- Reflection, serialization, and extension functions; |
||||
- PHP code executed by the ZendVM. |
||||
|
||||
### 3.3 Why It Must Be Kept |
||||
|
||||
Only Zend Object can fully carry PHP's runtime object semantics. Replacing it with Box would lose the class entry, object |
||||
handlers, visibility, Reflection, and dynamic dispatch; replacing it with Native Object would lose ZendVM visibility, |
||||
and force all dynamic behavior to degrade to compile-time restrictions. |
||||
|
||||
Ordinary PHP classes therefore always use Zend Object. The compiler can optimize calls, but cannot change its object model. |
||||
|
||||
## 4. PHPX Box |
||||
|
||||
### 4.1 Storage |
||||
|
||||
`php::Box` is a C++ polymorphic base class managed by PHPX. The Box pointer is registered as a Zend resource and carried by |
||||
`php::Var`: |
||||
|
||||
```text |
||||
zval(IS_RESOURCE) |
||||
-> zend_resource |
||||
-> php::Box* |
||||
-> concrete C++ value |
||||
``` |
||||
|
||||
The Zend resource's destructor callback ultimately calls `Box::destroy()`. Box can therefore pass through ordinary zval/Variant |
||||
call boundaries while hiding the concrete C++ type that Zend cannot express. |
||||
|
||||
Current main users include: |
||||
|
||||
- `StdContainerBox<std::vector<T>>`; |
||||
- `StdContainerBox<std::array<T, N>>`; |
||||
- `StdContainerBox<map-like type>`; |
||||
- High-precision values such as BigInt, BigFloat, and Decimal. |
||||
|
||||
### 4.2 The Std Container Hot Path |
||||
|
||||
Std Container local variables have a two-layer representation: |
||||
|
||||
```cpp |
||||
php::Var values = php::Var(new php::StdContainerBox<Container>(type_id)); |
||||
auto &values_ref = values.toBox<php::StdContainerBox<Container>>()->container; |
||||
``` |
||||
|
||||
`php::Var` is responsible for the lifecycle and necessary boundary passing; the concrete container reference is used for subsequent element access, avoiding re-extracting the Box on every operation. The container's key/value/length and other generic information are jointly saved by the compiler and the concrete C++ template type. |
||||
|
||||
When a Std Container is passed across TypePHP functions, the PHP function signature cannot express the following C++ type information: |
||||
|
||||
```text |
||||
std::vector<int> |
||||
std::vector<string> |
||||
std::map<string, App\User> |
||||
``` |
||||
|
||||
A PHP parameter can at most declare a non-generic class name or pseudo-type; it cannot simultaneously carry the container kind, key type, value |
||||
type, array dimensions, and length. The current approach uses `UnsafePtr`/`std::unsafe_cast()` with compiler type ID checking, |
||||
rather than generating every combination as a PHP class. |
||||
|
||||
In theory, parameter and return value annotations could be added to describe generics, but this would require maintaining extra metadata at every declaration, call, return, property, and propagation point, |
||||
and PHP Reflection still cannot fully express it. This standalone generic ABI is not being introduced for now. |
||||
|
||||
### 4.3 Box Boundaries |
||||
|
||||
Box is an opaque value carrier, not a general-purpose object system: |
||||
|
||||
- Zend GC only sees the resource and does not scan C++ references held inside Box; |
||||
- Box does not provide PHP class method tables, property tables, inheritance, or Reflection; |
||||
- The concrete type is recovered through `dynamic_cast`, type ID, or dedicated helpers; |
||||
- Box should not be used to build arbitrary cyclic object graphs that require bidirectional Zend/Box tracing; |
||||
- The usable locations and escape paths of Std Container continue to be restricted by the compiler. |
||||
|
||||
Box is suitable for numeric values, containers, and other extension values with clear boundaries. It is not suitable for replacing Native |
||||
business objects with arbitrary field reference relationships. |
||||
|
||||
### 4.4 Why It Must Be Kept |
||||
|
||||
The generic types of Std Container cannot be fully expressed by PHP function parameters; high-precision values in turn need to participate in existing operations and calls as |
||||
`php::Var`. Box provides all of the following: |
||||
|
||||
- A stable carrier that can be placed into a zval; |
||||
- Runtime recovery of the concrete C++ type; |
||||
- Automatic destruction within the Zend request lifecycle; |
||||
- A lightweight implementation that does not register a PHP class for each template instantiation. |
||||
|
||||
Zend Object cannot directly express C++ template instances; Native raw pointers cannot safely cross `php::Var` and dynamic |
||||
ZendVM boundaries. Therefore Box still has a reason to exist independently. |
||||
|
||||
## 5. Native Class Object |
||||
|
||||
### 5.1 Storage |
||||
|
||||
`#[Native]` classes do not register a Zend class, do not generate Zend object handlers, and have no zval representation. Each |
||||
object is a fixed-layout C++ struct in the Native Heap; TypePHP local variables, parameters, return values, and fields hold |
||||
concrete Native pointers: |
||||
|
||||
```cpp |
||||
php_app__point *point; |
||||
``` |
||||
|
||||
Methods continue to use TypePHP's free-function ABI: |
||||
|
||||
```cpp |
||||
php::Float php_app__point__length(php_app__point &this_); |
||||
``` |
||||
|
||||
Ordinary calls only pass a pointer value. No zval is created, no resource is registered, no reference counting is performed, and nothing goes through |
||||
`zend_call_function()`. |
||||
|
||||
### 5.2 Lifecycle |
||||
|
||||
Native Objects use an independent Wren-style precise, non-moving, stop-the-world mark-sweep GC in PHPX: |
||||
|
||||
- Native local variables, parameters, return temporaries, and global/static slots enter a precise root frame; |
||||
- The Native object descriptor is responsible for tracing Native pointer fields; |
||||
- When a Std Container saves a Native pointer, a dedicated container root frame is registered; |
||||
- Cyclic references are collected by the tracing GC, without relying on reference counts dropping to zero; |
||||
- The 16-byte GC header saves the minimal state required by the collector; |
||||
- `__destruct()` is executed by Native finalization, not by the Zend object destructor. |
||||
|
||||
Native pointer assignment does not increase the reference count and does not need a write barrier. Fixed fields are accessed directly by C++ offset. |
||||
|
||||
### 5.3 Passing Boundaries |
||||
|
||||
Native Object parameters and return values must explicitly declare a concrete Native class, or a supported nullable concrete type: |
||||
|
||||
```php |
||||
function distance(Point $left, Point $right): float; |
||||
function findPoint(): ?Point; |
||||
``` |
||||
|
||||
This lets the compiler generate the signature directly as `Point *`. Native Objects do not support: |
||||
|
||||
- Passing to PHP/ZendVM functions, Closures, or dynamic callables; |
||||
- Saving into PHP arrays, ordinary Zend Object properties, or `mixed`; |
||||
- Automatic conversion to `php::Object`, `php::Var`, or Interface value; |
||||
- Recovering the type through the runtime class name; |
||||
- Using the generic PHPX `toObject()` helper to complete boxing or unboxing. Native Classes can declare their own |
||||
`toObject(): object` method; keyword calls resolve directly to that Native Call and do not provide a generic bridge. |
||||
|
||||
When entering the PHP API, the user must explicitly convert the data, for example first calling Native `toArray(): array`, then passing |
||||
the result to `json_encode()`. This conversion produces a data copy and does not preserve the Native object identity. |
||||
|
||||
### 5.4 Why It Must Be Kept |
||||
|
||||
The goal of Native Class is hot-path performance close to C/C++: |
||||
|
||||
- A one-machine-word object handle; |
||||
- Fixed field layout; |
||||
- No Zend RC increment/decrement; |
||||
- No `zend_object` or `zend_resource` carrier allocation; |
||||
- Definite-symbol Native Calls; |
||||
- Inlinable and devirtualizable by the C++ compiler. |
||||
|
||||
If Box were used instead, each Native Object would need resource/zval wrapping, RC management, and concrete type recovery, and |
||||
Zend GC cannot scan the Native pointer graph inside Box; this both reduces performance and cannot correctly replace Native tracing |
||||
GC. If a custom `zend_object` were used instead, although it could connect to Zend GC and dynamic boundaries, the object header, RC, |
||||
handlers, and access paths would all change the performance positioning of Native Class. |
||||
|
||||
Therefore Native Class continues to use an independent Native Heap and a raw-pointer ABI. |
||||
|
||||
## 6. Why They Cannot Be Unified |
||||
|
||||
### 6.1 They Cannot All Become Zend Object |
||||
|
||||
This would unify dynamic semantics, but it would make Std Container generic instances and Native Class both bear the Zend object |
||||
header, RC, handlers, class registration, and dynamic access costs. Native Class would no longer be close to C/C++, |
||||
and Std Container would need a runtime class system designed for a large number of template combinations. |
||||
|
||||
### 6.2 They Cannot All Become Box |
||||
|
||||
Box can carry C++ values through zval, but Zend GC does not understand the object graph inside Box. It cannot replace the dynamic metadata of ordinary PHP |
||||
Object, nor can it provide a raw-pointer hot path while retaining Native cycle collection capability. |
||||
|
||||
### 6.3 They Cannot All Become Native Pointers |
||||
|
||||
Native pointers require complete static typing. Ordinary PHP objects need Reflection, dynamic properties, dynamic callables, |
||||
and Zend extension interop; the complete generic types of Std Container cannot be written into PHP parameter signatures. Turning these values into |
||||
raw pointers would produce type erasure that cannot be proven safe statically, and could lead to incorrect pointer conversion and crashes. |
||||
|
||||
### 6.4 No Automatic Bridging |
||||
|
||||
There is no implicit object identity conversion among the three models. Automatic boxing/unboxing would hide allocation, copying, RC, and GC root |
||||
changes, and would also make compiler boundaries no longer reliable. |
||||
|
||||
Allowed conversions must have clear semantics: |
||||
|
||||
- Std Container to PHP array: copies container data; |
||||
- Entity methods of Native Object such as `toArray()`: defined by the user and explicitly copy data; |
||||
- Explicit scalar conversion of high-precision types: produces new PHP scalar values; |
||||
- Ordinary Zend Object does not automatically become a Native Object. |
||||
|
||||
## 7. Compiler Implementation Constraints |
||||
|
||||
Future changes must preserve the following invariants: |
||||
|
||||
1. Determine the object model from the static type first, then choose the code generation path; never guess one of the three at runtime. |
||||
2. Native Objects must not be wrapped into `php::Var` or passed into the ZendVM due to a generic fallback. |
||||
3. Box concrete type recovery must validate the resource type and the concrete C++ type / type ID. |
||||
4. Zend Object optimization must not change Zend object identity, lifecycle, or dynamic visibility. |
||||
5. The argument ABIs of the three models must not be mixed: `php::Object`, Box-bearing `php::Var`, and `NativeClass *` |
||||
respectively represent different ownership and type constraints. |
||||
6. Cross-model conversions must be explicit, and the allocation or copying cost must be reflected in documentation and generated code. |
||||
7. If a new feature requires sacrificing all Native Class hot paths to gain a small amount of dynamic compatibility, it should be prohibited at compile time first. |
||||
8. If a new C++ generic type needs to cross the Zend value boundary, Box should be evaluated first, rather than widening the dynamic |
||||
boundary of Native Object. |
||||
9. If a value needs complete PHP object semantics, Zend Object should be used, and Box must not be treated as a simplified PHP class. |
||||
|
||||
## 8. Code Locations |
||||
|
||||
Main implementation entry points: |
||||
|
||||
```text |
||||
Ordinary Zend Object |
||||
compiler/src/Parser/* |
||||
phpx/include/phpx.h Object / Variant / Zend API wrappers |
||||
|
||||
PHPX Box and Std Container |
||||
phpx/include/phpx.h Box / StdContainerBox<T> |
||||
phpx/src/core/base.cc Box resource registration and destructor |
||||
compiler/src/Parser/StdContainerTrait.php |
||||
|
||||
Native Class Object |
||||
compiler/src/NativeClass/ |
||||
compiler/src/Transform/NativeClassAttributeLowering.php |
||||
phpx/include/phpx_native_gc.h |
||||
phpx/src/core/native_gc.cc |
||||
phpx/thirdparty/wren-gc/ |
||||
``` |
||||
|
||||
Detailed rules are in [STD_CONTAINERS.md](STD_CONTAINERS.md), |
||||
[NATIVE_CLASS_OBJECT.md](NATIVE_CLASS_OBJECT.md), and |
||||
[NATIVE_CLASS_IMPLEMENTATION_AUDIT.md](NATIVE_CLASS_IMPLEMENTATION_AUDIT.md). |
||||
|
||||
## 9. Current Decision |
||||
|
||||
The following refactorings are not being implemented at the current stage: |
||||
|
||||
- Not removing Wren GC; |
||||
- Not changing Native Object to Box or a custom Zend Object; |
||||
- Not adding a generic `toObject()` dynamic recovery mechanism to Native Object; the Native Class custom |
||||
`toObject(): object` remains an ordinary definite Native Call; |
||||
- Not changing Std Container to a raw-pointer ABI whose type cannot be expressed across signatures; |
||||
- Not attempting to cover the three object models with a single unified wrapper. |
||||
|
||||
These boundaries will be re-evaluated in the future only when the PHP language layer can stably express generic parameters, or when a new ABI |
||||
that has passed benchmark and complete GC correctness validation emerges. Until then, the coexistence of the three mechanisms is an intentional architectural choice. |
||||
@ -0,0 +1,406 @@ |
||||
# Patent Application Technical Disclosure: A Method for Implementing Strongly-Typed Data Containers in a Dynamic Language Using C++ Templates |
||||
|
||||
> This document is a draft technical disclosure for a patent application, intended to explain the technical solution to a patent agent. In this document, "the present invention" refers to "a method for implementing strongly-typed data containers in a dynamic language using C++ templates." This document does not constitute legal advice; the formal claims should be further drafted by a patent agent based on search results. |
||||
|
||||
## 1. Technical Application Product |
||||
|
||||
The present invention is applied to the Swoole-Compiler PHP AOT compiler. This product is used to pre-compile PHP dynamic-language programs into C++ native code, PHP extensions, or executable programs, while preserving PHP runtime compatibility in the compiled program. |
||||
|
||||
The present invention focuses on solving the problems of indeterminate array and container structure types, high runtime overhead, and difficulty in leveraging C++ static-type optimizations in dynamic languages. The solution introduces strongly-typed container declarations at the dynamic-language syntax level and converts them into C++ template container instances during the AOT compilation stage, enabling dynamic-language programs to achieve data-structure performance close to that of a static language in localized performance hotspots, while maintaining interoperability with dynamic-language arrays. |
||||
|
||||
## 2. Terminology |
||||
|
||||
| Term | English Explanation | |
||||
| --- | --- | |
||||
| PHP | A dynamically-typed scripting language | |
||||
| AOT | Ahead-Of-Time; compiling to target code before the program runs | |
||||
| C++ Template | A C++ mechanism for generating strongly-typed code at compile time | |
||||
| Dynamic language | A language in which variable types and function call targets can change at runtime | |
||||
| Strongly-typed container | A container whose key, value, length, nested structure, or class constraints are determined at compile time | |
||||
| PHP Array | The built-in array of the PHP language, which combines list, dictionary, and hash table semantics | |
||||
| zval | The internal structure used by the PHP runtime to store a value of any type | |
||||
| HashTable | The common underlying hash table structure of PHP Array | |
||||
| AST | Abstract Syntax Tree | |
||||
| Meta-information | Information recorded by the compiler about container kind, type, dimension, C++ declaration, etc. | |
||||
| Type identifier | An integer identifier used to distinguish different strongly-typed container structures | |
||||
| UnsafePtr | A controlled pointer wrapper that carries a container pointer and a type identifier | |
||||
|
||||
## 3. Technical Background and Existing Technical Solutions |
||||
|
||||
### 3.1 Broad Technical Background |
||||
|
||||
The advantage of dynamic languages lies in development flexibility. Taking PHP as an example, the same variable can hold an integer, a floating-point number, a string, an array, or an object in different places. The same PHP Array can act as a contiguous list, as a dictionary with string keys, and can simultaneously mix integer keys, string keys, and values of different types. |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
$data = []; |
||||
$data[] = 1; |
||||
$data["name"] = "swoole"; |
||||
$data[10] = new stdClass(); |
||||
``` |
||||
|
||||
This design lowers the barrier to business development, but it also makes it difficult for the compiler to determine the real shape of a data structure at compile time. For an AOT compiler, if the array element type, key type, length, and nested structure cannot be determined, it can only conservatively generate generic dynamic container code and cannot fully leverage C++ static typing and template optimization capabilities. |
||||
|
||||
In contrast, static languages such as C++, Rust, and Go generally require containers to have explicit types, for example `std::vector<int>` and `std::array<double, 100>`. Such containers have element sizes, access patterns, and memory layouts determined at compile time, so the access path is short and the compiler can further perform inlining, register allocation, and loop optimization. |
||||
|
||||
The present invention attempts to establish a localized strongly-typed data container mechanism between dynamic and static languages, so that dynamic-language code still keeps array-like syntax but is converted into C++ template containers at compile time. |
||||
|
||||
### 3.2 Narrow Technical Background |
||||
|
||||
Swoole-Compiler is a PHP AOT compiler. It parses PHP source files into an abstract syntax tree, then generates C++ code and compiles it into a PHP extension or a binary program. For ordinary PHP variables, the compiler can use runtime wrappers such as `php::Var` and `php::Array` to represent dynamic semantics. |
||||
|
||||
However, for scenarios such as high-frequency array access, numerical computation, fixed-length buffers, mapping tables, and object collections, continuing to use ordinary PHP Array incurs the following overhead: |
||||
|
||||
- Both keys and values require dynamic type judgment; |
||||
- Each value usually needs to be represented by a zval; |
||||
- Hash lookup, reference counting, copy-on-write, and other mechanisms lengthen the runtime path; |
||||
- The memory layout is not contiguous, resulting in poor CPU cache utilization; |
||||
- It is difficult for the compiler to confirm whether an array stores only one specific type; |
||||
- Passing large containers across functions tends to cause copying or dynamic wrapping overhead. |
||||
|
||||
Therefore, a strongly-typed container expression, checking, and code generation scheme suitable for an AOT compiler is needed. |
||||
|
||||
### 3.3 Closest Existing Technical Solutions |
||||
|
||||
Existing technologies can be roughly classified into the following categories: |
||||
|
||||
1. **Dynamic container solution using PHP Array exclusively** |
||||
All arrays are represented by the PHP runtime HashTable and zval. This solution offers good compatibility, but performance is limited by dynamic typing and the hash structure. |
||||
|
||||
2. **User-handwritten C++ extension solution** |
||||
Developers handwrite structures such as `std::vector` and `std::map` in C++ and expose them to PHP through PHP extension functions. This solution offers high performance, but the development cost is high, and users must manually maintain the type mapping between PHP and C++. |
||||
|
||||
3. **Generic JIT or AOT type inference solution** |
||||
The compiler attempts to infer the element types of a PHP Array from context. However, the dynamic writes, dynamic keys, function parameters, and return values of PHP Array make inference unstable, so optimization is usually only possible in very localized scenarios. |
||||
|
||||
4. **PHP userland container class solution** |
||||
For example, wrapping arrays or specific data structures through object classes. This solution improves the interface specification, but the underlying implementation may still rely on PHP objects, zval, and dynamic method calls, making it difficult to reach the performance level of C++ template containers. |
||||
|
||||
## 4. Shortcomings of Existing Technologies and Objectives of the Present Invention |
||||
|
||||
### 4.1 Shortcomings of Existing Technologies |
||||
|
||||
Existing solutions have the following shortcomings: |
||||
|
||||
1. The indeterminate types of dynamic arrays prevent the compiler from stably generating strongly-typed target code. |
||||
2. The underlying structure of ordinary PHP Array is too generic, incurring hash lookup and dynamic typing overhead in high-frequency access. |
||||
3. Handwritten C++ extensions require users to understand PHP extension development, memory management, and type conversion, raising the development barrier. |
||||
4. Generic type inference finds it hard to express complete container information such as fixed length, nested dimension, key type, and value class constraints. |
||||
5. There is a lack of unified interoperability rules between strongly-typed containers and PHP Array, easily splitting performance from compatibility. |
||||
6. When container references are passed across functions, directly exposing C++ pointers creates risks of type misuse and memory unsafety. |
||||
|
||||
### 4.2 Objectives of the Present Invention |
||||
|
||||
The objective of the present invention is to provide a method for implementing strongly-typed data containers in a dynamic language using C++ templates, so that dynamic-language programs can declare strongly-typed containers in localized code, the compiler generates C++ template container code, and automatic conversion to and from dynamic-language arrays is performed when necessary. |
||||
|
||||
Further, the present invention also provides a container reference passing mechanism carrying a type identifier, so that strongly-typed containers can be passed between Native functions with low copy cost while runtime type consistency checking is performed. |
||||
|
||||
## 5. Technical Solution of the Present Invention |
||||
|
||||
### 5.1 Overall Solution |
||||
|
||||
The present invention defines a set of strongly-typed container construction syntax in the dynamic language, for example: |
||||
|
||||
```php |
||||
$a = std::array(Type::Int, 100); |
||||
$v = std::vector(Type::Float); |
||||
$m = std::ordered_map(Type::String, Type::Int); |
||||
$h = std::map(Type::Int, User::class); |
||||
``` |
||||
|
||||
The compiler recognizes these construction expressions during the AOT stage, generates container meta-information, and converts them into C++ template instances: |
||||
|
||||
```cpp |
||||
php::StdArray<php::Int, 100> a{}; |
||||
php::StdVector<php::Float> v{}; |
||||
php::StdOrderedMap<php::Str, php::Int> m{}; |
||||
php::StdMap<php::Int, php::Object> h{}; |
||||
``` |
||||
|
||||
The compiler subsequently uses this meta-information for static checking and code generation during subscript access, assignment, iteration, function argument passing, and type conversion. |
||||
|
||||
### 5.2 System Composition |
||||
|
||||
```text |
||||
Figure 1: System composition diagram |
||||
|
||||
PHP source input module |
||||
| |
||||
v |
||||
Abstract syntax tree parsing module |
||||
| |
||||
v |
||||
Strongly-typed container recognition module |
||||
| |
||||
v |
||||
Container meta-information construction module |
||||
| |
||||
+--> Type identifier registration module |
||||
| |
||||
+--> Subscript access code generation module |
||||
| |
||||
+--> Assignment and copy determination module |
||||
| |
||||
+--> PHP Array interoperability module |
||||
| |
||||
+--> UnsafePtr auto-boxing and checking module |
||||
| |
||||
v |
||||
C++ template code generation module |
||||
| |
||||
v |
||||
C++ compiler |
||||
| |
||||
v |
||||
PHP extension or binary program |
||||
``` |
||||
|
||||
Each module is described as follows: |
||||
|
||||
- PHP source input module: reads dynamic-language source files. |
||||
- Abstract syntax tree parsing module: parses source code into syntax tree nodes. |
||||
- Strongly-typed container recognition module: recognizes container construction expressions such as `std::array`. |
||||
- Container meta-information construction module: records the container kind, key type, value type, class constraints, dimensions, etc. |
||||
- Type identifier registration module: generates a comparable type identifier for each container structure. |
||||
- Subscript access code generation module: generates array access, bounds checking, and key conversion code based on the container type. |
||||
- Assignment and copy determination module: determines whether to perform a C++ container copy or convert to PHP Array. |
||||
- PHP Array interoperability module: generates `php::toArray()` at dynamic semantic boundaries. |
||||
- UnsafePtr auto-boxing and checking module: generates container pointer wrappers carrying type identifiers at function call boundaries. |
||||
- C++ template code generation module: outputs strongly-typed C++ template container code. |
||||
|
||||
### 5.3 Method Flow |
||||
|
||||
```text |
||||
Figure 2: Method flow diagram |
||||
|
||||
Step S1: Parse the dynamic-language source code to obtain an abstract syntax tree; |
||||
Step S2: Recognize strongly-typed container construction expressions; |
||||
Step S3: Parse the container kind, key type, value type, class constraints, and dimensions; |
||||
Step S4: Generate container meta-information and register the type identifier; |
||||
Step S5: Record in the variable table that the variable is a strongly-typed container; |
||||
Step S6: When a subscript access is encountered, generate strongly-typed access code based on the container meta-information; |
||||
Step S7: When an assignment is encountered, determine whether the left and right sides are the same strongly-typed container; |
||||
Step S8: If they are the same, generate a C++ container copy; if not, report an error or convert to PHP Array according to the rules; |
||||
Step S9: When a Native function UnsafePtr parameter is encountered, auto-box the container pointer and type identifier; |
||||
Step S10: When the callee unboxes, check the type identifier; if it matches, return a C++ reference; otherwise, throw an exception. |
||||
``` |
||||
|
||||
### 5.4 Container Meta-information |
||||
|
||||
Container meta-information includes at least the following fields: |
||||
|
||||
```text |
||||
kind: container kind, for example array, vector, map, ordered_map; |
||||
decl: target C++ template declaration; |
||||
type: C++ type of the value; |
||||
class: class name when the value is an object; |
||||
keyType: key type of map or ordered_map; |
||||
sizes: dimension array of std::array; |
||||
bytes: estimated memory size of std::array; |
||||
typeId: type identifier generated from the above fields. |
||||
``` |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
$b = std::array(std::array(Type::Int, 3), 2); |
||||
``` |
||||
|
||||
The corresponding meta-information can be represented as: |
||||
|
||||
```text |
||||
kind=array |
||||
decl=php::StdArray<php::StdArray<php::Int, 3>, 2> |
||||
type=php::Int |
||||
sizes=[3, 2] |
||||
bytes=2 * 3 * sizeof(php::Int) |
||||
typeId=automatically assigned integer |
||||
``` |
||||
|
||||
### 5.5 std::array Nested Type Derivation |
||||
|
||||
`std::array` supports nested structures. The present invention derives the sub-array type based on the access level. |
||||
|
||||
Example: |
||||
|
||||
```php |
||||
$a = std::array(Type::Int, 3); |
||||
$b = std::array(std::array(Type::Int, 3), 2); |
||||
$a = $b[1]; |
||||
``` |
||||
|
||||
Processing method: |
||||
|
||||
1. The compiler reads the dimension information `[2, 3]` of `$b`. |
||||
2. Parse the access level of `$b[1]` as 1. |
||||
3. Compute the remaining dimensions `[3]`. |
||||
4. Derive the type of `$b[1]` as `std::array<int, 3>`. |
||||
5. Compare it with the type of `$a`. |
||||
6. If they are exactly the same, generate a C++ copy: |
||||
|
||||
```cpp |
||||
a = b[php::safeIndex(php::toInt(1L), 2)]; |
||||
``` |
||||
|
||||
Here `safeIndex` denotes the bounds-checking function. It ensures that dynamic-language subscript access still preserves out-of-bounds checking semantics. |
||||
|
||||
### 5.6 Same-type Copy and Dynamic Array Conversion |
||||
|
||||
The present invention divides assignment into two categories: |
||||
|
||||
The first category: the left value is a strongly-typed container and the right value is an exactly identical strongly-typed container: |
||||
|
||||
```php |
||||
$a = std::vector(Type::Int); |
||||
$b = std::vector(Type::Int); |
||||
$a = $b; |
||||
``` |
||||
|
||||
Generated: |
||||
|
||||
```cpp |
||||
a = b; |
||||
``` |
||||
|
||||
The second category: the left value is an ordinary dynamic variable and the right value is a strongly-typed container: |
||||
|
||||
```php |
||||
$arr = $a; |
||||
``` |
||||
|
||||
Generated: |
||||
|
||||
```cpp |
||||
arr = php::toArray(a); |
||||
``` |
||||
|
||||
This rule keeps C++ performance within strongly-typed regions, and automatically turns strongly-typed containers into ordinary PHP Array when they flow into dynamic-language regions. |
||||
|
||||
### 5.7 Subscript Access and Writing |
||||
|
||||
For `std::vector`: |
||||
|
||||
```php |
||||
$v[] = 1; |
||||
$v[0] = 2; |
||||
``` |
||||
|
||||
Generated similarly to: |
||||
|
||||
```cpp |
||||
v.push_back(php::toInt(1L)); |
||||
v.offsetSet(php::toInt(0L), php::toInt(2L)); |
||||
``` |
||||
|
||||
For `std::map`: |
||||
|
||||
```php |
||||
$m["x"] = 10; |
||||
``` |
||||
|
||||
Generated similarly to: |
||||
|
||||
```cpp |
||||
m.offsetSet(php::toString("x"), php::toInt(10L)); |
||||
``` |
||||
|
||||
For class-typed values: |
||||
|
||||
```php |
||||
$v = std::vector(User::class); |
||||
$v[] = new User(); |
||||
``` |
||||
|
||||
The compiler checks whether the written object is of the specified class, avoiding mixing incorrect objects into the container. |
||||
|
||||
### 5.8 foreach Iteration |
||||
|
||||
Strongly-typed container iteration is compiled into a C++ iterator loop: |
||||
|
||||
```php |
||||
foreach ($v as $i => $value) { |
||||
// ... |
||||
} |
||||
``` |
||||
|
||||
Generated similarly to: |
||||
|
||||
```cpp |
||||
for (auto it = v.begin(); it != v.end(); ++it) { |
||||
i = it - v.begin(); |
||||
value = *it; |
||||
} |
||||
``` |
||||
|
||||
For map types, the key is obtained from `it->first` and the value from `it->second`. |
||||
|
||||
### 5.9 UnsafePtr Auto-boxing and Type Checking |
||||
|
||||
To support low-copy passing of containers between Native functions, the present invention provides the `UnsafePtr` parameter mechanism. |
||||
|
||||
User code: |
||||
|
||||
```php |
||||
function update(UnsafePtr $ptr): void |
||||
{ |
||||
$v = std::unsafe_cast(std::vector(Type::Int), $ptr); |
||||
$v[0] = 100; |
||||
} |
||||
|
||||
function main(): void |
||||
{ |
||||
$v = std::vector(Type::Int, 1); |
||||
update($v); |
||||
} |
||||
``` |
||||
|
||||
Caller-side generation: |
||||
|
||||
```cpp |
||||
php_update(php_create_unsafe_ptr(&v, typeId)); |
||||
``` |
||||
|
||||
Callee-side generation: |
||||
|
||||
```cpp |
||||
auto &v = php_unsafe_cast<php::StdVector<php::Int>>(ptr, typeId); |
||||
``` |
||||
|
||||
Where `UnsafePtr` holds: |
||||
|
||||
```cpp |
||||
void *ptr; |
||||
uint32_t type_id; |
||||
``` |
||||
|
||||
When unboxing, `type_id` is compared; a C++ reference is returned only if it matches, otherwise a type exception is thrown. This avoids incorrectly converting `std::vector<int>` to `std::vector<float>`. |
||||
|
||||
## 6. Key Points and Points to Protect |
||||
|
||||
1. Recognize strongly-typed container construction syntax during the AOT compilation of a dynamic language, and convert it into C++ template container instances. |
||||
2. Use a container meta-information table to uniformly record the container kind, C++ declaration, key type, value type, class constraints, dimensions, memory size, and type identifier. |
||||
3. Derive the sub-array type of nested `std::array` based on the access level, and support C++ copy between sub-arrays and same-type containers. |
||||
4. Determine assignment semantics based on the left and right container meta-information: if exactly the same, generate a C++ copy; otherwise, convert to a dynamic array or report a compile-time error. |
||||
5. Generate strongly-typed C++ access code for subscript access while preserving the dynamic language's bounds-checking or key type conversion semantics. |
||||
6. Generate a C++ iterator loop for foreach while preserving the dynamic language's key/value iteration form. |
||||
7. At Native function call boundaries, auto-box strongly-typed containers into UnsafePtr carrying type identifiers based on ArgInfo. |
||||
8. Perform runtime checking based on the type identifier during unboxing, and return a C++ container reference only after the check passes. |
||||
|
||||
## 7. Advantages Compared with Existing Technologies |
||||
|
||||
Compared with the ordinary PHP Array solution, the present invention can determine the container structure and element types at compile time and generate C++ template instances, thereby reducing dynamic type judgment, hash lookup, and zval wrapping overhead. |
||||
|
||||
Compared with the handwritten C++ extension solution, developers still use syntax close to PHP arrays; container declaration, type checking, subscript access, copy, iteration, dynamic array conversion, and cross-function reference passing are all completed automatically by the compiler, lowering the development barrier. |
||||
|
||||
Compared with ordinary type inference solutions, the present invention does not rely on guessing how a PHP Array is used; instead, it establishes stable type meta-information through explicit strongly-typed container syntax, making the compilation result more predictable. |
||||
|
||||
## 8. Optional Implementations |
||||
|
||||
The present invention is not limited to the PHP language; it can also be used in AOT compilers for Python, JavaScript, Ruby, and other dynamic languages. As long as a dynamic-language compiler can recognize strongly-typed container declarations and generate C++, Rust, Go, or other static-language target code, a similar technical solution can be adopted. |
||||
|
||||
The C++ template containers in the present invention are also not limited to `StdArray`, `StdVector`, `StdOrderedMap`, and `StdMap`; they can be extended to strongly-typed containers such as queues, sets, ring buffers, matrices, and tensors. |
||||
|
||||
## 9. Confidentiality Statement |
||||
|
||||
This document involves the internal compiler implementation of Swoole-Compiler, the design of strongly-typed container meta-information, the UnsafePtr type identifier mechanism, and code generation strategies. Before formal filing, it is recommended that it be managed as internal technical material. |
||||
@ -0,0 +1,453 @@ |
||||
# Patent Application Technical Disclosure: A Method and System for Compilation and Execution of a Hybrid-State Programming Language |
||||
|
||||
> This document is a draft technical disclosure for a patent application, explaining the technical solution to a patent agent. "The present invention" herein refers to "a method and system for compilation and execution of a hybrid-state programming language". This document does not constitute legal advice; the formal claims should be further drafted by a patent agent in combination with search results. |
||||
|
||||
## 1. Technical Application Product |
||||
|
||||
The present invention is applied to the Swoole-Compiler PHP AOT compiler. Swoole-Compiler neither simply fully staticizes PHP, nor is it a pure interpreter or pure JIT compiler in the traditional sense, but rather a "hybrid-state programming language compilation and execution system". |
||||
|
||||
In this system, the same PHP program can simultaneously contain: |
||||
|
||||
- Static-state code that can be statically analyzed and compiled ahead of time; |
||||
- Dynamic-state code that needs to preserve PHP's dynamic semantics; |
||||
- Boundary transition code between static state and dynamic state; |
||||
- Entry points that can be called by the dynamic runtime as a PHP extension; |
||||
- Entry points that can be executed as a standalone binary program. |
||||
|
||||
This system enables a dynamic language to gain the performance advantages of a static language while preserving the flexibility advantages of a dynamic language. |
||||
|
||||
## 2. Terminology |
||||
|
||||
| Term | Description | |
||||
| --- | --- | |
||||
| Hybrid state | A single program simultaneously contains static compilation state and dynamic interpretation state, and can switch between the two states | |
||||
| Static state | A state where the compiler can determine types, functions, methods, properties, or control paths at compile time | |
||||
| Dynamic state | A state where types, functions, methods, or properties must be resolved at runtime according to PHP semantics | |
||||
| AOT | Ahead-Of-Time compilation | |
||||
| VM | Virtual Machine; here it refers to the PHP Zend VM | |
||||
| Native function | A PHP function or method compiled by AOT into a C++ function | |
||||
| Dynamic call | Looking up and calling a target at runtime based on function name, method name, object type, or callback object | |
||||
| Static direct link | Determining the call target at compile time and generating a direct C++ function call | |
||||
| ArgInfo | Parameter information, including type, default value, reference, variadic parameters, etc. | |
||||
| Symbol table | A data structure in which the compiler records information about functions, classes, interfaces, traits, constants, properties, methods, etc. | |
||||
| Wrapper function | A bridging function that converts a Zend VM call entry point into a Native function call | |
||||
| Fallback | Falling back to the dynamic runtime path when static analysis cannot guarantee semantic correctness | |
||||
|
||||
## 3. Technical Background and Existing Technical Solutions |
||||
|
||||
### 3.1 Broad Technical Background |
||||
|
||||
Traditional programming languages can generally be divided into two categories. |
||||
|
||||
The first category is statically compiled languages, such as C, C++, Go, and Rust. They complete compilation before running, and types, function signatures, and memory layouts are mostly determined at compile time, resulting in high runtime performance. However, such languages generally have weaker development flexibility, with limited support for dynamic loading, dynamic method calls, and runtime modification of data structures. |
||||
|
||||
The second category is dynamically interpreted languages, such as PHP, Python, Ruby, and JavaScript. They resolve variable types, function calls, object properties, and method dispatch at runtime, resulting in high development efficiency and flexible expressiveness. However, such languages generally require virtual machines, interpreters, dynamic type determination, and runtime lookups, and their performance is inferior to statically compiled languages. |
||||
|
||||
Existing JIT technologies attempt to compile hot code into machine code at runtime, but JIT still relies on runtime sampling, type feedback, and hot-spot detection, and it is difficult to obtain fully deterministic native code before deployment. Traditional AOT technologies tend to fully staticize a program, but they easily lose language semantics when facing dynamic-language features such as variable functions, dynamic methods, magic methods, reflection, dynamic properties, closures, and callbacks. |
||||
|
||||
### 3.2 Narrow Technical Background |
||||
|
||||
Swoole-Compiler targets PHP programs. PHP programs have the following dynamic characteristics: |
||||
|
||||
- Variables can hold any type; |
||||
- Functions and methods can be called dynamically through strings or arrays; |
||||
- Objects can handle dynamic behavior through magic methods such as `__call()`, `__get()`, `__set()`; |
||||
- Namespaces, use aliases, traits, inheritance, and method overrides affect the actual call target; |
||||
- Parameters support default values, named arguments, variadic parameters, and reference parameters; |
||||
- PHP built-in functions and user functions can be called interchangeably; |
||||
- A program can run either as a PHP extension or as a standalone binary program. |
||||
|
||||
If the compiler forcibly converts all PHP code into static C++ calls, it will break the dynamic semantics described above. If dynamic interpreted execution is fully preserved, the performance advantages of AOT cannot be obtained. |
||||
|
||||
Therefore, a new hybrid-state programming language design is needed: enter static state in regions that can be statically determined, enter dynamic state in regions that cannot, and achieve interoperation between the two states through a unified boundary mechanism. |
||||
|
||||
### 3.3 Closest Existing Technical Solutions |
||||
|
||||
Existing technologies include: |
||||
|
||||
1. **Pure interpreted execution** |
||||
PHP source code is interpreted and executed by the Zend VM. This solution has strong compatibility, but every function call, property access, and array operation relies on runtime dynamic mechanisms. |
||||
|
||||
2. **Pure static compilation** |
||||
The entire source program is converted to C/C++ or machine code. This solution has high performance, but has difficulty supporting dynamic language features, usually requiring significant syntax restrictions or changes to language semantics. |
||||
|
||||
3. **JIT compilation** |
||||
Compilation is performed at runtime based on hot paths and type feedback. This solution can improve hot-spot performance, but still requires interpreter cooperation, and the compilation result depends on runtime state. |
||||
|
||||
4. **Handwritten extensions or FFI** |
||||
Dynamic languages call static code through C/C++ extensions or foreign function interfaces. This solution can partially improve performance, but lacks a unified language-level compilation model between static and dynamic code. |
||||
|
||||
## 4. Drawbacks of Existing Technologies and Objectives of the Present Invention |
||||
|
||||
### 4.1 Drawbacks of Existing Technologies |
||||
|
||||
Existing solutions have the following problems: |
||||
|
||||
1. Pure interpreted execution cannot fully exploit static types and native code performance. |
||||
2. Pure static compilation has difficulty being compatible with dynamic language features such as dynamic calls, magic methods, dynamic properties, and reflection. |
||||
3. JIT relies on runtime hot spots and type feedback, making it difficult to obtain stable, predictable compilation artifacts before deployment. |
||||
4. Handwritten extensions require developers to explicitly maintain interfaces, type conversions, and lifetimes between dynamic and static languages. |
||||
5. Function, class, property, and method lookups in dynamic languages usually rely on strings, and repeated lookups are costly. |
||||
6. Traditional compilers often use "whether it can be fully staticized" as the criterion, lacking a language-level model expressing the coexistence of multiple execution states within the same program. |
||||
|
||||
### 4.2 Objectives of the Present Invention |
||||
|
||||
The objective of the present invention is to provide a method and system for compilation and execution of a hybrid-state programming language, so that the same dynamic language program can be divided into static state and dynamic state: |
||||
|
||||
- In static state, the compiler generates C++ direct calls, strongly typed variables, strongly typed containers, and native code; |
||||
- In dynamic state, the system preserves the PHP VM's dynamic lookup, dynamic calls, dynamic properties, callbacks, and general zval semantics; |
||||
- Between the two states, interoperation is achieved through ArgInfo, wrapper functions, symbol caching, type conversion, and dynamic fallback mechanisms. |
||||
|
||||
In this way, the performance advantages of a static language and the flexibility advantages of a dynamic language are both obtained. |
||||
|
||||
## 5. Technical Solution of the Present Invention |
||||
|
||||
### 5.1 Hybrid-State Language Model |
||||
|
||||
The present invention proposes a "hybrid-state" language model. This model does not simply compile a dynamic language into a static language; instead, with the joint support of the compiler and the runtime, it divides statements, expressions, functions, methods, variables, and calls in a program into different states. |
||||
|
||||
The hybrid state includes at least: |
||||
|
||||
1. **Static function state**: function definitions, parameter types, and return types can be determined, generating C++ Native functions. |
||||
2. **Static object state**: the class of an object variable can be determined, and method calls can be directly linked to C++ functions. |
||||
3. **Static data state**: variables can be mapped to C++ native types or C++ template containers. |
||||
4. **Dynamic value state**: variables use `php::Var`, zval, or PHP objects for storage, preserving dynamic type semantics. |
||||
5. **Dynamic call state**: when the function name, method name, or object type cannot be determined, calls are made through the PHP runtime. |
||||
6. **Boundary bridging state**: between static and dynamic states, argument extraction, type conversion, return value write-back, and symbol caching are performed. |
||||
|
||||
### 5.2 System Composition |
||||
|
||||
```text |
||||
Figure 1: Hybrid-state programming language system composition diagram |
||||
|
||||
PHP source code / project configuration |
||||
| |
||||
v |
||||
Preprocessing and symbol extraction module |
||||
| |
||||
v |
||||
Dependency ordering and semantic analysis module |
||||
| |
||||
v |
||||
Hybrid-state determination module |
||||
| |
||||
+--> Static-state code generation module |
||||
| | |
||||
| +--> C++ function direct link |
||||
| +--> C++ native types |
||||
| +--> C++ template containers |
||||
| |
||||
+--> Dynamic-state code generation module |
||||
| | |
||||
| +--> PHP VM dynamic calls |
||||
| +--> zval / php::Var dynamic values |
||||
| +--> magic methods and callbacks |
||||
| |
||||
+--> Boundary bridging module |
||||
| |
||||
+--> ArgInfo argument conversion |
||||
+--> Zend wrapper functions |
||||
+--> symbol caching |
||||
+--> dynamic fallback |
||||
| |
||||
v |
||||
C++ source code and extension registration code |
||||
| |
||||
v |
||||
PHP extension or executable binary |
||||
``` |
||||
|
||||
### 5.3 Method Flow |
||||
|
||||
```text |
||||
Figure 2: Hybrid-state compilation and execution flow diagram |
||||
|
||||
Step S1: Read PHP source files and project configuration; |
||||
Step S2: Parse the abstract syntax tree; |
||||
Step S3: Extract symbols such as functions, classes, interfaces, traits, constants, properties, and methods; |
||||
Step S4: Order files by dependency based on symbol usage relationships; |
||||
Step S5: Resolve the ArgInfo of functions and methods; |
||||
Step S6: Determine for each expression or call site whether it can enter static state; |
||||
Step S7: If the target, types, and arguments can be determined, generate C++ static direct link code; |
||||
Step S8: If there is a risk of dynamic semantics, generate dynamic-state call code; |
||||
Step S9: Generate argument conversion, return value conversion, and wrapper functions for the boundary between static and dynamic states; |
||||
Step S10: Generate runtime cache mappings for functions, classes, methods, and properties; |
||||
Step S11: Compile the C++ code to produce a PHP extension or executable program; |
||||
Step S12: At runtime, execute the static direct link or dynamic fallback path according to the generated code. |
||||
``` |
||||
|
||||
### 5.4 Preprocessing and Symbol Extraction |
||||
|
||||
Before formally generating C++ code, the compiler first scans the PHP source code and extracts: |
||||
|
||||
- Function definitions; |
||||
- Class definitions; |
||||
- Interface definitions; |
||||
- Trait definitions; |
||||
- Class properties; |
||||
- Class methods; |
||||
- Class constants; |
||||
- Global constants; |
||||
- Namespaces and use aliases; |
||||
- Usage relationships among functions, classes, methods, and constants. |
||||
|
||||
It then builds a symbol dependency graph and performs topological sorting on the source files. This improves the success rate of static analysis and avoids functions or classes being temporarily invisible due to file ordering. |
||||
|
||||
### 5.5 ArgInfo Unified Parameter Model |
||||
|
||||
The present invention uses ArgInfo to describe function and method parameters, including: |
||||
|
||||
```text |
||||
Parameter name; |
||||
Parameter type; |
||||
Object class name; |
||||
Whether it is a reference parameter; |
||||
Whether it is a variadic parameter; |
||||
Whether it is nullable; |
||||
Default value; |
||||
Whether it is an UnsafePtr; |
||||
Whether it is a constructor property promotion parameter. |
||||
``` |
||||
|
||||
ArgInfo serves two directions simultaneously: |
||||
|
||||
1. **AOT internal direct-link calls**: the compiler reorders named arguments, fills default values, merges variadic parameters, and generates C++ direct calls based on ArgInfo. |
||||
2. **Zend runtime entry calls**: when a compiled function is called by the PHP VM as a PHP extension function, the wrapper function reads parameters from the call stack and converts types according to ArgInfo. |
||||
|
||||
This design allows static and dynamic states to share the same set of parameter semantics. |
||||
|
||||
### 5.6 Static-State Code Generation |
||||
|
||||
When the compiler can determine the call target, it generates a C++ direct call. |
||||
|
||||
Example: |
||||
|
||||
```php |
||||
function add(int $a, int $b): int |
||||
{ |
||||
return $a + $b; |
||||
} |
||||
|
||||
add(1, 2); |
||||
``` |
||||
|
||||
Generates something like: |
||||
|
||||
```cpp |
||||
php_add(php::toInt(1L), php::toInt(2L)); |
||||
``` |
||||
|
||||
For object methods: |
||||
|
||||
```php |
||||
$obj->run($arg); |
||||
``` |
||||
|
||||
If the class of `$obj` can be determined, and there is no method override or dynamic magic method risk, it generates: |
||||
|
||||
```cpp |
||||
php_Class_run(obj, converted_arg); |
||||
``` |
||||
|
||||
Static state also includes: |
||||
|
||||
- Native types such as `int`, `float`, `bool`; |
||||
- C++ template containers such as `std::array`, `std::vector`; |
||||
- Property access on objects whose class can be determined; |
||||
- Default argument and variadic parameter handling for functions that can be determined; |
||||
- Constant and class constant access that can be determined at compile time. |
||||
|
||||
### 5.7 Dynamic-State Code Generation |
||||
|
||||
The compiler enters dynamic state in the following cases: |
||||
|
||||
- The function name comes from a variable; |
||||
- The method name comes from a variable; |
||||
- The actual type of an object cannot be determined; |
||||
- The class may have a `__call()` magic method; |
||||
- A method is overridden by a subclass, and static direct link may change dynamic dispatch semantics; |
||||
- A PHP built-in dynamic function needs to be called; |
||||
- Callbacks, closures, or placeholder expressions cannot be fully determined statically; |
||||
- Variables use general `mixed` or dynamic array semantics. |
||||
|
||||
Dynamic state generates something like: |
||||
|
||||
```cpp |
||||
php::call(function_ptr, arg_list); |
||||
``` |
||||
|
||||
or: |
||||
|
||||
```cpp |
||||
object.call(method_ptr, arg_list); |
||||
``` |
||||
|
||||
Dynamic state preserves the PHP VM's function lookup, method dispatch, zval types, and dynamic callback semantics. |
||||
|
||||
### 5.8 Static Direct Link vs. Dynamic Fallback Determination |
||||
|
||||
The present invention does not require all code to be staticized; instead, a determination is made at each call site. |
||||
|
||||
```text |
||||
Figure 3: Call-site state determination flow |
||||
|
||||
Call expression |
||||
| |
||||
v |
||||
Is the function name or method name a literal? |
||||
| |
||||
+-- No --> dynamic-state call |
||||
| |
||||
+-- Yes |
||||
| |
||||
v |
||||
Can a Native target be found in the symbol table? |
||||
| |
||||
+-- No --> dynamic-state call |
||||
| |
||||
+-- Yes |
||||
| |
||||
v |
||||
Can the object class or function signature be determined? |
||||
| |
||||
+-- No --> dynamic-state call |
||||
| |
||||
+-- Yes |
||||
| |
||||
v |
||||
Are there risks such as magic methods, overrides, or dynamic callbacks? |
||||
| |
||||
+-- Yes --> dynamic-state call |
||||
| |
||||
+-- No --> static-state direct link |
||||
``` |
||||
|
||||
This mechanism means the language runtime is not a single state, but automatically selects the best state by code location. |
||||
|
||||
### 5.9 Boundary Bridging Mechanism |
||||
|
||||
Static and dynamic states interoperate through a boundary bridging mechanism. |
||||
|
||||
#### 5.9.1 Dynamic Call to Static Function |
||||
|
||||
When the PHP VM calls a compiled function, it enters a Zend wrapper function: |
||||
|
||||
```text |
||||
Zend call entry point |
||||
| |
||||
v |
||||
Read call stack arguments |
||||
| |
||||
v |
||||
Convert arguments according to ArgInfo |
||||
| |
||||
v |
||||
Call the Native function |
||||
| |
||||
v |
||||
Write the return value back to return_value |
||||
``` |
||||
|
||||
#### 5.9.2 Static Code Calling Dynamic Function |
||||
|
||||
When AOT code cannot determine the target, it generates a dynamic call: |
||||
|
||||
```text |
||||
C++ static code |
||||
| |
||||
v |
||||
Construct a PHP dynamic argument list |
||||
| |
||||
v |
||||
Look up the function or method |
||||
| |
||||
v |
||||
Call the PHP VM dynamic path |
||||
| |
||||
v |
||||
Return a php::Var dynamic value |
||||
``` |
||||
|
||||
#### 5.9.3 Static Data Flowing into Dynamic State |
||||
|
||||
For example, when a strongly typed container is assigned to a normal PHP variable, it is automatically converted to a PHP Array: |
||||
|
||||
```php |
||||
$arr = $vector; |
||||
``` |
||||
|
||||
Generates: |
||||
|
||||
```cpp |
||||
arr = php::toArray(vector); |
||||
``` |
||||
|
||||
#### 5.9.4 Dynamic Object Flowing into Static State |
||||
|
||||
When an object is obtained from a dynamic array or function return value, and a subsequent static method call is needed, the compiler can be informed of the object's class name through a compile-time type declaration function or a type conversion function, thereby entering static object state. |
||||
|
||||
### 5.10 Symbol Caching Mechanism |
||||
|
||||
To reduce the runtime string lookup overhead of dynamic state, the present invention establishes integer IDs and cache arrays for symbols: |
||||
|
||||
```cpp |
||||
zend_class_entry *class_map[N]; |
||||
zend_function *func_map[M]; |
||||
uint32_t property_map[K]; |
||||
``` |
||||
|
||||
On first lookup, the Zend structure pointer is obtained by string name and cached; subsequent calls access it quickly through the integer ID. |
||||
|
||||
Example: |
||||
|
||||
```cpp |
||||
zend_function *get_func(int id, const php::Str &name) { |
||||
if (func_map[id] == nullptr) { |
||||
func_map[id] = php::getFunction(name); |
||||
} |
||||
return func_map[id]; |
||||
} |
||||
``` |
||||
|
||||
This mechanism makes dynamic state still more efficient than the pure interpreted path. |
||||
|
||||
### 5.11 Dual Runtime Forms |
||||
|
||||
The present invention supports two runtime forms: |
||||
|
||||
1. **Extension form**: the compilation result is a PHP extension that can be loaded into PHP-FPM or CLI and called by the PHP VM. |
||||
2. **Binary form**: the compilation result is an executable program, entered through the `main()` function and directly runnable. |
||||
|
||||
These two forms share the same hybrid-state compilation model. |
||||
|
||||
## 6. Key Points and Points to Protect |
||||
|
||||
1. Propose a hybrid-state programming language model that allows static state and dynamic state to coexist in the same dynamic language program. |
||||
2. Automatically determine static or dynamic state in the compiler by function, method, variable, expression, and call site. |
||||
3. Generate C++ static direct link code for determinable targets, and automatically fall back to PHP dynamic call code for undeterminable targets. |
||||
4. Use ArgInfo as the parameter semantic model shared by static and dynamic states. |
||||
5. Generate Zend wrapper functions so the dynamic runtime can call AOT-generated Native functions. |
||||
6. Construct dynamic argument lists in AOT static code so static code can call PHP dynamic functions, dynamic methods, or callbacks. |
||||
7. Use a symbol caching mechanism to reduce the string lookup overhead of functions, classes, methods, and properties in dynamic-state runtime. |
||||
8. Support two runtime artifacts, extension form and binary form, both based on the same hybrid-state language model. |
||||
9. Support native types and C++ template containers in static state, and preserve zval, php::Var, PHP Array, and PHP object semantics in dynamic state. |
||||
|
||||
## 7. Advantages over Existing Technologies |
||||
|
||||
Compared with traditional statically compiled languages, the present invention preserves the dynamic call, dynamic type, dynamic object, and callback capabilities of dynamic languages, without requiring full program staticization. |
||||
|
||||
Compared with traditional dynamically interpreted languages, the present invention can generate C++ direct calls, native types, and strongly typed containers in regions determinable at compile time, significantly reducing runtime lookup and dynamic type overhead. |
||||
|
||||
Compared with JIT, the present invention generates stable compilation artifacts before deployment, does not rely on runtime hot-spot sampling and type feedback, and is suitable for production environments that require stable deployment, source code protection, and performance improvement. |
||||
|
||||
Compared with handwritten extensions, the present invention automatically generates wrapper functions, argument conversion, symbol caching, and dynamic fallback paths through the compiler, reducing the cost of manually bridging between PHP and C++. |
||||
|
||||
## 8. Alternative Embodiments |
||||
|
||||
The present invention is not limited to PHP; it can also be applied to dynamic languages such as Python, JavaScript, Ruby, and Lua. The target static language is not limited to C++; it can also be Rust, Go, C, or LLVM intermediate representation. |
||||
|
||||
The granularity of hybrid-state determination can be function-level, basic-block-level, statement-level, or expression-level. Symbol caching can also be implemented using hash tables, arrays, handle tables, or runtime inline caching. |
||||
|
||||
## 9. Confidentiality Statement |
||||
|
||||
This document involves the overall compilation model of Swoole-Compiler, the static direct link and dynamic fallback strategy, the ArgInfo boundary bridging mechanism, and the symbol caching solution. Before formal application, it is recommended to manage it as internal technical material. |
||||
@ -0,0 +1,355 @@ |
||||
# Rebuilding the PHPX WASM Static Library |
||||
|
||||
This document is aimed at TypePHP/PHPX developers and explains how to recompile and install the |
||||
PHPX static library for `wasm32-wasip2`. Ordinary TypePHP users do not need to perform these steps; release packages should provide the complete |
||||
WASI SDK directly. |
||||
|
||||
## Directory Convention |
||||
|
||||
This document assumes the source layout is as follows: |
||||
|
||||
```text |
||||
/home/swoole/workspace/aot/ |
||||
├── compiler/ |
||||
└── phpx/ |
||||
``` |
||||
|
||||
It is recommended to set the PHPX root directory first: |
||||
|
||||
```shell |
||||
export PHPX_HOME=/home/swoole/workspace/aot/phpx |
||||
``` |
||||
|
||||
The installation prefix is fixed to: |
||||
|
||||
```text |
||||
$PHPX_HOME/wasm/wasm32-wasip2 |
||||
``` |
||||
|
||||
This directory is both the input of the existing PHP/WASI SDK and the installation location of the PHPX build results: |
||||
|
||||
```text |
||||
wasm/wasm32-wasip2/ |
||||
├── include/php/ PHP/WASI headers |
||||
├── include/phpx/ PHPX/TypePHP runtime headers |
||||
├── lib/libphp.a |
||||
├── lib/libphpx.a |
||||
├── lib/libgmp.a |
||||
├── lib/libgmpxx.a |
||||
├── lib/libmpfr.a |
||||
├── lib/libmpdec.a |
||||
├── lib/libmpdec++.a |
||||
└── .typephp-wasi-sdk-abi |
||||
``` |
||||
|
||||
Do not copy host-platform `libphpx.so`, `phpx.dll`, or `.a` files here. |
||||
WASM static libraries contain the target ABI and cannot be used across WASI, Linux, macOS, or Windows. |
||||
|
||||
## Toolchain Preparation |
||||
|
||||
PHPX WASM currently supports only WASI 0.2 Preview 2. Add the WASI SDK to `PATH`: |
||||
|
||||
```shell |
||||
export PATH=/opt/wasi-sdk-33.0/bin:$PATH |
||||
``` |
||||
|
||||
`PATH` only lets the shell and build tools find the WASI SDK programs; it does not make CMake automatically select the |
||||
WASI target. When configuring the build directory for the first time, you must still pass |
||||
`-DCMAKE_TOOLCHAIN_FILE=.../wasi-sdk-p2.cmake`. If you omit it, CMake will choose the host platform's |
||||
`/usr/bin/cc` and `/usr/bin/c++`, and PHPX's target check will immediately reject that configuration. |
||||
|
||||
Confirm the necessary tools: |
||||
|
||||
```shell |
||||
command -v wasm32-wasip2-clang |
||||
command -v wasm32-wasip2-clang++ |
||||
command -v llvm-ar |
||||
command -v llvm-ranlib |
||||
command -v llvm-nm |
||||
command -v cmake |
||||
command -v ninja |
||||
``` |
||||
|
||||
Confirm the compile target: |
||||
|
||||
```shell |
||||
wasm32-wasip2-clang++ --print-target-triple |
||||
``` |
||||
|
||||
It must output: |
||||
|
||||
```text |
||||
wasm32-unknown-wasip2 |
||||
``` |
||||
|
||||
The installation prefix must already contain PHP/WASI headers and `libphp.a` matching the current PHPX: |
||||
|
||||
```shell |
||||
test -f "$PHPX_HOME/wasm/wasm32-wasip2/include/php/main/php.h" |
||||
test -f "$PHPX_HOME/wasm/wasm32-wasip2/lib/libphp.a" |
||||
``` |
||||
|
||||
## Daily Development: Rebuild PHPX Directly with CMake |
||||
|
||||
When a PHPX `.cc` or header file changes, incrementally rebuild directly using `phpx/wasm/CMakeLists.txt`. |
||||
This is the recommended flow for daily development; it does not re-download or recompile PHP, GMP, or MPFR, nor does it regenerate |
||||
`libphp.a`. |
||||
|
||||
First, locate the CMake toolchain from the current WASI compiler to avoid depending on a hardcoded SDK version path: |
||||
|
||||
```shell |
||||
WASI_RESOURCE_DIR="$(wasm32-wasip2-clang++ --print-resource-dir)" |
||||
WASI_SDK_ROOT="$(cd "$WASI_RESOURCE_DIR/../../.." && pwd)" |
||||
WASI_CMAKE_TOOLCHAIN="$WASI_SDK_ROOT/share/cmake/wasi-sdk-p2.cmake" |
||||
test -f "$WASI_CMAKE_TOOLCHAIN" |
||||
``` |
||||
|
||||
### Using Ninja (Recommended) |
||||
|
||||
First configure a persistent build directory: |
||||
|
||||
```shell |
||||
cmake \ |
||||
-S "$PHPX_HOME/wasm" \ |
||||
-B "$PHPX_HOME/build/wasm32-wasip2" \ |
||||
-G Ninja \ |
||||
-DCMAKE_TOOLCHAIN_FILE="$WASI_CMAKE_TOOLCHAIN" \ |
||||
-DCMAKE_BUILD_TYPE=Release \ |
||||
-DPHPX_WASI_SDK_DIR="$PHPX_HOME/wasm/wasm32-wasip2" \ |
||||
-DCMAKE_INSTALL_PREFIX="$PHPX_HOME/wasm/wasm32-wasip2" |
||||
``` |
||||
|
||||
The toolchain takes effect when CMake executes `project()`, so it can only be set during the first configuration of a build directory. |
||||
If the directory was configured without a toolchain before and has already cached the host compiler, do not append arguments onto the existing cache; |
||||
use a new build directory instead, for example: |
||||
|
||||
```shell |
||||
cmake \ |
||||
-S "$PHPX_HOME/wasm" \ |
||||
-B "$PHPX_HOME/build/wasm32-wasip2-wasi" \ |
||||
-G Ninja \ |
||||
-DCMAKE_TOOLCHAIN_FILE="$WASI_CMAKE_TOOLCHAIN" \ |
||||
-DCMAKE_BUILD_TYPE=Release \ |
||||
-DPHPX_WASI_SDK_DIR="$PHPX_HOME/wasm/wasm32-wasip2" \ |
||||
-DCMAKE_INSTALL_PREFIX="$PHPX_HOME/wasm/wasm32-wasip2" |
||||
``` |
||||
|
||||
Subsequent build/install commands should also use this new directory. |
||||
|
||||
Compile and install: |
||||
|
||||
```shell |
||||
cmake --build "$PHPX_HOME/build/wasm32-wasip2" --parallel 16 |
||||
cmake --install "$PHPX_HOME/build/wasm32-wasip2" |
||||
``` |
||||
|
||||
When the PHPX source changes again later, you only need to run: |
||||
|
||||
```shell |
||||
cmake --build "$PHPX_HOME/build/wasm32-wasip2" --parallel 16 |
||||
cmake --install "$PHPX_HOME/build/wasm32-wasip2" |
||||
``` |
||||
|
||||
CMake/Ninja only recompiles the changed source files and then updates `libphpx.a` in the installation directory. |
||||
|
||||
### Using Make |
||||
|
||||
`make` can be used, but the `Unix Makefiles` generator must be selected during the first configuration, using a different build |
||||
directory; you cannot switch generators in a directory already configured by Ninja: |
||||
|
||||
```shell |
||||
cmake \ |
||||
-S "$PHPX_HOME/wasm" \ |
||||
-B "$PHPX_HOME/build/wasm32-wasip2-make" \ |
||||
-G "Unix Makefiles" \ |
||||
-DCMAKE_TOOLCHAIN_FILE="$WASI_CMAKE_TOOLCHAIN" \ |
||||
-DCMAKE_BUILD_TYPE=Release \ |
||||
-DPHPX_WASI_SDK_DIR="$PHPX_HOME/wasm/wasm32-wasip2" \ |
||||
-DCMAKE_INSTALL_PREFIX="$PHPX_HOME/wasm/wasm32-wasip2" |
||||
|
||||
make -C "$PHPX_HOME/build/wasm32-wasip2-make" -j16 |
||||
make -C "$PHPX_HOME/build/wasm32-wasip2-make" install |
||||
``` |
||||
|
||||
After later modifying PHPX code, just repeat the two `make` commands. You can also use the generator-independent form: |
||||
|
||||
```shell |
||||
cmake --build "$PHPX_HOME/build/wasm32-wasip2-make" --parallel 16 |
||||
cmake --install "$PHPX_HOME/build/wasm32-wasip2-make" |
||||
``` |
||||
|
||||
The artifacts of Ninja and Make are the same; Ninja is generally faster in dependency scanning and incremental builds, so internal development defaults |
||||
to Ninja. |
||||
|
||||
This flow updates: |
||||
|
||||
- `lib/libphpx.a` |
||||
- `lib/libmpdec.a` and `lib/libmpdec++.a` (recompiled only when the related source changes) |
||||
- PHPX public headers under `include/phpx/` |
||||
- `.typephp-wasi-runtime-abi` |
||||
|
||||
It does not update `libphp.a`, GMP, or MPFR, nor does it rewrite the full SDK's |
||||
`.typephp-wasi-sdk-abi`. Therefore, this flow should be run on an already fully installed SDK. |
||||
|
||||
### Force Recompiling PHPX |
||||
|
||||
When you suspect that old objects or the CMake cache are no longer trustworthy, prefer using a new, explicit build directory: |
||||
|
||||
```shell |
||||
cmake \ |
||||
-S "$PHPX_HOME/wasm" \ |
||||
-B "$PHPX_HOME/build/wasm32-wasip2-clean" \ |
||||
-G Ninja \ |
||||
-DCMAKE_TOOLCHAIN_FILE="$WASI_CMAKE_TOOLCHAIN" \ |
||||
-DCMAKE_BUILD_TYPE=Release \ |
||||
-DPHPX_WASI_SDK_DIR="$PHPX_HOME/wasm/wasm32-wasip2" \ |
||||
-DCMAKE_INSTALL_PREFIX="$PHPX_HOME/wasm/wasm32-wasip2" |
||||
|
||||
cmake --build "$PHPX_HOME/build/wasm32-wasip2-clean" --parallel 16 |
||||
cmake --install "$PHPX_HOME/build/wasm32-wasip2-clean" |
||||
``` |
||||
|
||||
This does not delete `libphp.a` and the dependency libraries in the installation directory, nor does it mix in old CMake configuration. |
||||
|
||||
## First Build or Rebuilding PHPX Numeric Dependencies |
||||
|
||||
Use PHPX's unified build entry in the following cases: |
||||
|
||||
- Setting up the PHPX WASI installation directory for the first time; |
||||
- GMP or MPFR version, patch, or compile parameter changes; |
||||
- Changes to PHPX vendored mpdecimal or its WASI configuration; |
||||
- The need to check and install all PHPX WASI headers and static libraries at once. |
||||
|
||||
```shell |
||||
cd "$PHPX_HOME" |
||||
|
||||
./wasm/build.sh \ |
||||
--prefix "$PHPX_HOME/wasm/wasm32-wasip2" \ |
||||
--build-dir "$PHPX_HOME/build/wasm32-wasip2-sdk" \ |
||||
--jobs 16 |
||||
``` |
||||
|
||||
Explicitly use `$PHPX_HOME/build/` to avoid the default `/tmp` build directory being lost after a reboot. The downloaded GMP and |
||||
MPFR source and build cache are retained and can be reused in subsequent builds. |
||||
|
||||
This entry builds or installs: |
||||
|
||||
- `libphpx.a` |
||||
- `libgmp.a`, `libgmpxx.a` |
||||
- `libmpfr.a` |
||||
- `libmpdec.a`, `libmpdec++.a` |
||||
- The corresponding headers and the PHPX runtime ABI marker |
||||
|
||||
It requires PHP/WASI headers to already exist in the installation prefix; it does not build `libphp.a`. |
||||
|
||||
## PHP ABI Changes: Rebuilding the Full SDK |
||||
|
||||
If the PHP source, extension set, PHP configuration, Zend ABI, or PHP installed headers change, you must rebuild the full SDK from the |
||||
TypePHP compiler repository, and you cannot replace only `libphpx.a`: |
||||
|
||||
```shell |
||||
cd /home/swoole/workspace/aot/compiler |
||||
|
||||
./wasm/build-sdk.sh \ |
||||
--prefix "$PHPX_HOME/wasm/wasm32-wasip2" \ |
||||
--php-source "$PWD/projects/php-8.5.9" \ |
||||
--phpx-source "$PHPX_HOME" \ |
||||
--build-dir "$PWD/build/wasm-sdk" \ |
||||
--jobs 16 |
||||
``` |
||||
|
||||
The full build installs the PHP and PHPX parts in sequence, and writes the following after all artifacts are verified: |
||||
|
||||
```text |
||||
.typephp-wasi-sdk-abi |
||||
``` |
||||
|
||||
Do not forge this marker by hand. The existence of the marker only means the build flow declares ABI compatibility; it cannot fix actually mixed |
||||
old headers or static libraries. |
||||
|
||||
## Artifact Verification |
||||
|
||||
After installation completes, check the key files: |
||||
|
||||
```shell |
||||
WASI_PREFIX="$PHPX_HOME/wasm/wasm32-wasip2" |
||||
|
||||
test -s "$WASI_PREFIX/lib/libphpx.a" |
||||
test -s "$WASI_PREFIX/lib/libphp.a" |
||||
test -f "$WASI_PREFIX/include/phpx/phpx.h" |
||||
test -f "$WASI_PREFIX/include/phpx/phpx_helper.h" |
||||
test -f "$WASI_PREFIX/include/phpx/typephp_helper.h" |
||||
|
||||
llvm-ar t "$WASI_PREFIX/lib/libphpx.a" | head |
||||
cat "$WASI_PREFIX/.typephp-wasi-runtime-abi" |
||||
cat "$WASI_PREFIX/.typephp-wasi-sdk-abi" |
||||
``` |
||||
|
||||
The current markers should be: |
||||
|
||||
```text |
||||
typephp-wasip2-phpx-abi-v1 |
||||
typephp-wasip2-sdk-abi-v4 |
||||
``` |
||||
|
||||
The marker versions will be upgraded as the ABI design evolves; if the expected values in the code have changed, follow the current build scripts |
||||
rather than writing old values back just to pass detection. |
||||
|
||||
## TypePHP Regression Verification |
||||
|
||||
First verify the Wasmtime component: |
||||
|
||||
```shell |
||||
cd /home/swoole/workspace/aot/compiler |
||||
|
||||
PHPX_HOME="$PHPX_HOME" \ |
||||
./run-tests.php --wasm --compiler ./bin/tpc.php tests/wasm/ |
||||
``` |
||||
|
||||
Then verify that Wasmtime and Chrome output are consistent, and cover parallel build/output directory isolation: |
||||
|
||||
```shell |
||||
PHPX_HOME="$PHPX_HOME" \ |
||||
./run-tests.php -j 4 --target wasm-all --compiler ./bin/tpc.php tests/wasm/ |
||||
``` |
||||
|
||||
Browser tests also require `jco`, Node.js, and Chrome to be in `PATH`. `wasm-all` runs each case in |
||||
Wasmtime and Chrome separately, and compares the output of both sides. |
||||
|
||||
Finally build the browser example: |
||||
|
||||
```shell |
||||
cd examples/wasm-hello |
||||
PHPX_HOME="$PHPX_HOME" ../../bin/tpc.php project.yml |
||||
npm run build |
||||
``` |
||||
|
||||
## Common Errors |
||||
|
||||
### `PersistentCacheSlot` or PHPX helpers are undefined |
||||
|
||||
The generated code uses a new PHPX header/API, but `include/phpx/` or |
||||
`lib/libphpx.a` in the installation prefix is still an old version. Run the "Daily development: rebuild PHPX only" flow, and make sure the configuration and installation |
||||
use the same `PHPX_WASI_SDK_DIR`/`CMAKE_INSTALL_PREFIX`. |
||||
|
||||
### `TypePHP WASI SDK is missing or ABI-incompatible` |
||||
|
||||
Check whether `PHPX_HOME` points to the actual PHPX root directory, and whether the full SDK marker, PHP/PHPX headers, |
||||
and static libraries come from the same compatible build. Run the full SDK rebuild when the PHP ABI has changed. |
||||
|
||||
### CMake detects the host compiler |
||||
|
||||
You must pass the WASI SDK's `wasi-sdk-p2.cmake`. Do not use the host |
||||
`CMakeLists.txt` in the PHPX root directory to build WASM directly. Adding the WASI SDK to `PATH` is not equivalent to loading the CMake |
||||
toolchain. If `CMakeCache.txt` has already recorded `/usr/bin/cc` or `/usr/bin/c++`, use a |
||||
new build directory to reconfigure. |
||||
|
||||
### TypePHP still links the old implementation after modifying PHPX |
||||
|
||||
Confirm that `PHPX_HOME` takes priority over the Composer directory, and check the actual artifact time: |
||||
|
||||
```shell |
||||
stat "$PHPX_HOME/wasm/wasm32-wasip2/lib/libphpx.a" |
||||
``` |
||||
|
||||
TypePHP should read both headers and static libraries from the same `$PHPX_HOME/wasm/wasm32-wasip2`. |
||||
@ -0,0 +1,223 @@ |
||||
# PHP 8.4 Property Hook Integration Design |
||||
|
||||
This document records how the TypePHP compiler and PHPX implement PHP 8.4 Property Hooks, focusing on Zend metadata registration, object introspection, memory lifetime, and version compatibility boundaries. This is an internal maintenance document; user-facing syntax documentation belongs in the external documentation repository. |
||||
|
||||
A Property Hook without an implementation body in an Interface is an abstract property contract and does not go through the concrete-class lowering flow described here. For its model, variance checking, and Zend metadata registration, see [Interface Property Hook Implementation Plan](INTERFACE_PROPERTY_HOOKS.md). |
||||
|
||||
## 1. Background |
||||
|
||||
TypePHP compiles Property Hook bodies into hidden AOT getters/setters. Doing only this step satisfies property reads/writes that the compiler explicitly identifies, but ZendVM does not know that these hidden methods represent Property Hooks, so the following dynamic capabilities diverge from PHP 8.4: |
||||
|
||||
- `ReflectionProperty::hasHooks()`, `getHooks()`, and `isVirtual()`; |
||||
- `get_object_vars()`, `json_encode()`, and `var_export()`; |
||||
- `foreach` traversal of objects; |
||||
- the storage difference between backed properties and virtual properties; |
||||
- dynamic property reads/writes initiated by ZendVM. |
||||
|
||||
TypePHP does not emulate these PHP behaviors separately. The compiler preserves Hook metadata after lowering, and PHPX wires the AOT methods into the native PHP 8.4 Property Hook structures when the class is registered during MINIT. After that, Reflection and object introspection reuse the standard ZendVM implementation. |
||||
|
||||
## 2. Compilation Flow |
||||
|
||||
### 2.1 AST lowering |
||||
|
||||
`PropertyHookLowering` converts each Hook into a hidden class method and records on the property AST: |
||||
|
||||
- the hidden method names corresponding to the getter/setter; |
||||
- whether the Hook accesses its own backing storage; |
||||
- whether the property is a virtual property. |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
public string $name { |
||||
get => strtoupper($this->name); |
||||
set => $this->name = trim($value); |
||||
} |
||||
``` |
||||
|
||||
This produces equivalent hidden getters/setters internally. `$this->name` inside the Hook is marked as backing access to avoid recursion by calling the Hook again. |
||||
|
||||
If the Hook does not access backing storage, the property is marked virtual. This conclusion must be obtained during the lowering stage, because the generated Zend property declaration needs it to decide whether to allocate a property slot. |
||||
|
||||
### 2.2 Class registration code |
||||
|
||||
After `gen_stub.php` declares the property and obtains the `zend_property_info *`, it generates: |
||||
|
||||
```cpp |
||||
typephp_register_property_hooks( |
||||
class_entry, |
||||
property_info, |
||||
getter_method_name, |
||||
setter_method_name |
||||
); |
||||
``` |
||||
|
||||
The call happens during the class's persistent registration stage, not on the request hot path. |
||||
|
||||
## 3. PHPX Registration Flow |
||||
|
||||
PHPX's `typephp_register_property_hooks()` is implemented only for PHP 8.4 and above, and lives in a TypePHP-specific helper. |
||||
|
||||
### 3.1 Locating the AOT implementation method |
||||
|
||||
PHPX finds the hidden method produced by lowering from the class method table: |
||||
|
||||
```cpp |
||||
zend_hash_str_find_ptr(&ce->function_table, method_name.data(), method_name.size()); |
||||
``` |
||||
|
||||
This method is an already-registered `zend_internal_function` whose handler ultimately enters the TypePHP-generated C++ getter/setter. The lookup happens only once; property reads/writes do not re-query the function table. |
||||
|
||||
### 3.2 Creating the Hook function descriptor |
||||
|
||||
The hidden function object in the class method table cannot be directly modified or reused. Zend Property Hooks require an independent function identity and property association: |
||||
|
||||
```cpp |
||||
hook->function_name = "$name::get"; // or "$name::set" |
||||
hook->prop_info = property_info; |
||||
``` |
||||
|
||||
PHPX therefore copies a `zend_internal_function` descriptor and replaces the Hook-specific fields. The copy does not produce a second C++ implementation; the handler, argument info, and other persistent data still come from the original AOT method. |
||||
|
||||
An independent function descriptor avoids breaking the class method table's key, reflection name, or ownership relationships when the hidden method is modified, and lets Reflection correctly report `$name::get` and `$name::set`. |
||||
|
||||
### 3.3 Mounting the property Hook |
||||
|
||||
PHP 8.4 added a Hook table to `zend_property_info`: |
||||
|
||||
```cpp |
||||
property_info->hooks[ZEND_PROPERTY_HOOK_GET] = getter; |
||||
property_info->hooks[ZEND_PROPERTY_HOOK_SET] = setter; |
||||
``` |
||||
|
||||
The following must also be updated: |
||||
|
||||
```cpp |
||||
ce->num_hooked_props++; |
||||
``` |
||||
|
||||
Zend's Reflection, object property construction, and inheritance checks all read this metadata. Registering only the hidden method without filling in `property_info->hooks` will not be recognized by Zend as a true Property Hook. |
||||
|
||||
### 3.4 Installing the Hook object iterator |
||||
|
||||
When the class has no custom iterator, PHPX sets: |
||||
|
||||
```cpp |
||||
ce->get_iterator = zend_hooked_object_get_iterator; |
||||
``` |
||||
|
||||
`zend_hooked_object_get_iterator()` is the `ZEND_API` exported by PHP 8.4 in `zend_property_hooks.h`. PHP itself installs this iterator when compiling classes that contain Property Hooks. |
||||
|
||||
The ordinary object iterator mainly traverses physical property slots, while the Hook iterator is also responsible for: |
||||
|
||||
- calling getters for backed and virtual properties; |
||||
- skipping virtual properties without a getter; |
||||
- enforcing property visibility rules; |
||||
- rejecting unsupported by-reference traversal; |
||||
- merging dynamic properties. |
||||
|
||||
Therefore PHPX should not duplicate a traversal implementation. Reusing Zend's exported implementation keeps `foreach` behavior consistent and reduces maintenance cost. |
||||
|
||||
## 4. Virtual property |
||||
|
||||
PHP 8.4 uses a special offset to represent a virtual property: |
||||
|
||||
```cpp |
||||
#define ZEND_VIRTUAL_PROPERTY_OFFSET ((uint32_t) -1) |
||||
``` |
||||
|
||||
When Zend declares a property, it needs `IS_UNDEF` as the declaration value to establish a virtual offset for a property with `ZEND_ACC_VIRTUAL`. Therefore the generated code uses: |
||||
|
||||
```cpp |
||||
zval default_value; |
||||
ZVAL_UNDEF(&default_value); |
||||
``` |
||||
|
||||
It must not be replaced with `null` or an ordinary default value, otherwise Zend may allocate a backing slot and `ReflectionProperty::isVirtual()` will return the wrong result. |
||||
|
||||
## 5. Object introspection and serialization |
||||
|
||||
When `ce->num_hooked_props` is nonzero, Zend's `zend_std_get_properties_for()` calls `zend_hooked_object_build_properties()` in scenarios such as JSON, `get_object_vars()`, and `var_export()`. That function reads the Hooked public property values. |
||||
|
||||
Serialization uses different semantics: |
||||
|
||||
- a virtual property has no persistent state and does not appear in the serialization result; |
||||
- a backed property serializes the backing value, not the value computed by the getter; |
||||
- private storage properties are still serialized according to PHP's property-name mangling rules. |
||||
|
||||
This difference is existing PHP 8.4 behavior and must not be overridden just to make JSON and serialization output identical. |
||||
|
||||
## 6. Lifetime and thread safety |
||||
|
||||
TypePHP AOT classes are registered as persistent internal classes. The Hook table, Hook function descriptors, and function names must share the same process-level lifetime, so PHPX uses: |
||||
|
||||
```cpp |
||||
pemalloc(size, true); |
||||
zend_string_init(data, length, true); |
||||
``` |
||||
|
||||
Request memory must not be used; otherwise the class entry would keep dangling pointers after RSHUTDOWN and the next request could crash when accessing properties or Reflection. |
||||
|
||||
Registration happens only in MINIT: |
||||
|
||||
- during request execution, Hook metadata is read-only; |
||||
- it does not need to be rebuilt on every request; |
||||
- it does not need to look up hidden methods on every property access; |
||||
- NTS has no locking overhead; |
||||
- under ZTS, registration completes before worker threads process requests, so the class entry is not concurrently modified. |
||||
|
||||
## 7. PHP version boundary |
||||
|
||||
The minimum version of both TypePHP and PHPX is PHP 8.4, so the Property Hook implementation directly uses the following PHP 8.4 ABI: |
||||
|
||||
- `zend_property_info::hooks`; |
||||
- `zend_class_entry::num_hooked_props`; |
||||
- `ZEND_PROPERTY_HOOK_*`; |
||||
- `ZEND_PROPERTY_HOOK_STRUCT_SIZE`; |
||||
- `ZEND_VIRTUAL_PROPERTY_OFFSET`; |
||||
- `zend_hooked_object_get_iterator()`. |
||||
|
||||
PHPX headers and CMake configuration reject headers/`php-config` below PHP 8.4. PHP 8.4 and 8.5 still build separate PHPX binaries; `--php-version` only controls source syntax and does not require exact minor-version parity with `libphp.so`, but both must be no lower than 8.4. |
||||
|
||||
## 8. ABI risk and upgrade checks |
||||
|
||||
`zend_hooked_object_get_iterator()` is an exported Zend API, but Property Hooks overall remain a version-dependent low-level Zend ABI. PHP 8.4 does not provide a complete high-level `zend_declare_property_hook()` extension API, so the current implementation must fill in Zend metadata. |
||||
|
||||
The rationale for this approach is: |
||||
|
||||
1. TypePHP and PHPX are version-locked and recompiled against specific PHP versions; |
||||
2. the registration flow matches the steps Zend's compiler performs for native Property Hooks; |
||||
3. only Zend's exported iterator is reused, without duplicating its complex implementation; |
||||
4. versions below PHP 8.4 are uniformly rejected at the build entry point; |
||||
5. all registration completes in MINIT, adding no name lookup to the request hot path. |
||||
|
||||
When upgrading the PHP version, the following must be checked: |
||||
|
||||
1. whether the Hook fields and ownership of `zend_property_info` changed; |
||||
2. whether `ZEND_PROPERTY_HOOK_COUNT` and Hook kinds increased; |
||||
3. whether virtual property declaration conditions and offsets changed; |
||||
4. whether `zend_hooked_object_get_iterator()` is still an exported API; |
||||
5. whether class linking, inheritance, variance, and Reflection added new required metadata; |
||||
6. whether the destruction and inheritance-copy rules for persistent internal functions changed. |
||||
|
||||
If Zend later provides an official extension registration API, migration to that API should be prioritized to reduce direct dependence on internal structure layout. |
||||
|
||||
## 9. Test requirements |
||||
|
||||
Property Hook changes must at least cover: |
||||
|
||||
- direct getter/setter and backing access; |
||||
- Reflection differences between virtual properties and backed properties; |
||||
- `hasHooks()`, `getHooks()`, Hook names, and final status; |
||||
- `get_object_vars()`, JSON, and object `foreach`; |
||||
- serialization containing only real stored state; |
||||
- dynamic Zend property reads/writes; |
||||
- inheritance and property visibility; |
||||
- PHP 8.4 and PHP 8.5 builds. |
||||
|
||||
Current core regression tests are located at: |
||||
|
||||
- `tests/compiler/object_property/property-hooks.phpt`; |
||||
- `tests/compiler/object_property/property-hooks-operations.phpt`; |
||||
- `tests/compiler/object_property/property-hooks-reflection.phpt`; |
||||
- `tests/compiler/object_property/property-hooks-introspection.phpt`. |
||||
@ -0,0 +1,151 @@ |
||||
# py2php: Python → TypePHP source conversion tool |
||||
|
||||
## Usage |
||||
|
||||
```bash |
||||
./bin/tpc.php --convert-python-to-php examples/python/version.py > examples/python/version.php |
||||
``` |
||||
|
||||
The generated PHP source is written to stdout, errors to stderr, with exit code 0 for success / 1 for failure. |
||||
|
||||
## Architecture |
||||
|
||||
``` |
||||
.py source |
||||
└─ PythonAstLoader python3 subprocess (ast module) → JSON AST |
||||
└─ PythonToTypePhpConverter AST → TypePHP source string |
||||
└─ Command::execute CLI dispatch (--convert-python-to-php) |
||||
``` |
||||
|
||||
- Source: `src/PythonTools/Command.php`, `src/PythonTools/Converter/` |
||||
- Unsupported syntax throws `RuntimeException("{file}:{line}: unsupported Python syntax {node type}[: details]")`, which the CLI layer converts to stderr + exit code 1. |
||||
- Tests: `phpunit/src/PythonTools/` (`PythonToTypePhpConverterTest`, `PythonAstLoaderTest`, `PythonToolsCommandTest`), corresponding item by item to this document. |
||||
|
||||
## Statement support matrix |
||||
|
||||
| Python syntax | Status | Conversion rule / error | |
||||
|---|---|---| |
||||
| `x = expr` | ✅ | `$x = expr;`, module-level variables are automatically injected as `global` | |
||||
| `x = y = 1` (chained assignment) | ✅ | `$x = $y = 1;` (name targets only; errors on property/subscript targets) | |
||||
| `x += expr` and other augmented assignments | ✅ | supports the `+ - * / % ** << >> \| ^ &` families; `//=` `@=` expand to `python\operator\floordiv/matmul($x, ...)` calls | |
||||
| `x: int = expr` | ✅ | annotation ignored, converted to an ordinary assignment | |
||||
| `x: int` (annotation only) | ✅ | converted to comment `// annotation-only declaration: x`, not registered as a module global | |
||||
| `a, b = x` (destructuring) | ✅ | `[$a, $b] = $x->toArray();` (PyObject converted to PHP array then destructured; elements may be names/properties/subscripts. Nested destructuring, star destructuring `a, *b = x`, and chained destructuring are not supported. Element-count mismatches fill `null` per PHP semantics rather than raising Python's ValueError) | |
||||
| `def f(...)` | ✅ | see "Function signatures"; a function named `main` is renamed to `main_` (to avoid conflict with the TypePHP entry point), and call sites are rewritten accordingly | |
||||
| nested `def` | ❌ | `FunctionDef: nested functions require Python closure scope analysis` | |
||||
| `@decorator` | ✅ | see "Function decorators" | |
||||
| `return [expr]` | ✅ | `return [expr];` | |
||||
| `if / elif / else` | ✅ | isomorphic conversion | |
||||
| `while` | ✅ | isomorphic conversion; `while/else` is not supported | |
||||
| `for i in iter` | ✅ | `foreach (iter as $i)`; `for/else` and tuple targets are not supported | |
||||
| `break` / `continue` / `pass` | ✅ | `pass` → `// pass` comment | |
||||
| `global x` | ✅ | `global $x;` (when combined with the auto-injected global it appears twice — redundant but valid, a known behavior) | |
||||
| `del x` / `del o.a` / `del d[k]` | ✅ | `unset(...)`; `del (a, b)` tuple/list targets expanded item by item; invalid del targets (such as `del f()`) are rejected first by the Python parser | |
||||
| module-level string literal (docstring) | ✅ | converted to `/** ... */` comment (`*/` escaped as `* /`) | |
||||
| `import a.b` | ✅ | `use python\a;` (only the first segment as the alias, see "Known behaviors") | |
||||
| `import a.b as x` | ✅ | `use python\a\b as x;` (`as` omitted when the alias equals the last segment) | |
||||
| `from m import f [as g]` | ✅ | call sites mapped to `python\m\f(...)` | |
||||
| `from . import m` | ❌ | `ImportFrom: relative imports are not supported yet` | |
||||
| `from m import *` | ❌ | `ImportFrom: star imports are not supported` | |
||||
| `class` | ❌ | `ClassDef` | |
||||
| `with` | ❌ | `With` | |
||||
| `raise` / `try` / `assert` | ❌ | `Raise` / `Try` / `Assert` | |
||||
| `async def` / `await` | ❌ | `AsyncFunctionDef` (`await` unreachable, outer level errors first) | |
||||
| `match` | ❌ | `Match` | |
||||
| `nonlocal` | ❌ | `Nonlocal` | |
||||
|
||||
## Function signatures |
||||
|
||||
| Python form | Status | TypePHP output | |
||||
|---|---|---| |
||||
| `def f(x, y=4)` | ✅ | `function f($x, $y = 4)` | |
||||
| `def f(a, *, b)` | ✅ | `function f($a, $b = null)` (keyword-only parameters without defaults are padded with `null`) | |
||||
| `def f(*args)` / `def f(**kw)` | ✅ | `function f(...$args)` | |
||||
| `def f(*a, **kw)` | ❌ | `FunctionDef: simultaneous *args and **kwargs cannot be represented by one PHP signature` | |
||||
| `lambda a, b=2: a + b` | ✅ | `fn ($a, $b = 2) => $a + $b` | |
||||
|
||||
## Expression support matrix |
||||
|
||||
| Python syntax | Status | Conversion rule / error | |
||||
|---|---|---| |
||||
| literals `int / float / str / True / False / None` | ✅ | `var_export`; `None` → `null` | |
||||
| `b'...'` bytes | ❌ | `{file}: Python bytes literals are not supported yet` (no line number) | |
||||
| `1j` complex | ❌ | `{file}: Python complex literals are not supported yet` (no line number) | |
||||
| variable names | ✅ | `$name`; `this` escaped as `$this_` | |
||||
| module alias as a value | ❌ | `a Python module cannot be used as a first-class value in TypePHP namespace syntax` | |
||||
| attribute chain `o.a.b` | ✅ | `$o->a->b`; for module alias chains only the first segment is a module member: `sys.version_info.major` → `sys\version_info->major` | |
||||
| module attribute assignment/deletion | ❌ | `Attribute: Python module attributes cannot be assigned or deleted` | |
||||
| function call | ✅ | defined functions connect directly `f(...)`; built-ins mapped `python\len(...)`; `from m import f` mapped `python\m\f(...)`; other names callable as variables `$f(...)` | |
||||
| keyword arguments / `*args` / `**kwargs` calls | ✅ | `f(x: 1, ...$args)` | |
||||
| container literals `[] () {} {:}` | ✅ | `python\list/tuple/set/dict([...])`, supports `...` unpacking | |
||||
| binary operators `+ - * / % ** << >> \| ^ &` | ✅ | isomorphic conversion | |
||||
| `//` floor division / `@` matrix multiplication | ✅ | `python\operator\floordiv(a, b)` / `python\operator\matmul(a, b)` | |
||||
| unary operators `- + not ~` | ✅ | `- + ! ~` | |
||||
| comparisons `== != < <= > >=` | ✅ | isomorphic conversion | |
||||
| `is` / `is not` | ✅ | `===` / `!==` | |
||||
| `in` / `not in` | ✅ | `python\operator\contains(b, a)` (arguments swapped) / negated | |
||||
| chained comparison `a < b < c` | ❌ | `Compare: chained comparisons require explicit temporary variables` | |
||||
| `a and b` / `a or b` | ❌ | `BoolOp` | |
||||
| `x if c else y` | ✅ | `(c ? x : y)` | |
||||
| subscript `a[i]` / slice `a[l:u:s]` | ✅ | `$a[$i]` / `$a[python\slice(l, u, s)]` (defaults to `null`) | |
||||
| f-string | ✅ | concatenation + `->toString()`; operator-precedence-sensitive expressions are parenthesized as a whole | |
||||
| f-string `!r` conversion / `:03d` format spec | ❌ | `FormattedValue: formatted f-string conversions are not supported yet` | |
||||
| walrus `:=` | ✅ | assignment within expression `($n = 10)` | |
||||
| comprehensions / generator expressions | ❌ | `ListComp` / `SetComp` / `DictComp` / `GeneratorExp` | |
||||
| `yield` / `yield from` | ❌ | `Yield` / `YieldFrom` | |
||||
|
||||
## Function decorators |
||||
|
||||
Decorators rebind the function to the same-named module variable at the start of `main()` (before other top-level statements), bottom-up per Python semantics: |
||||
|
||||
```python |
||||
@a |
||||
@b |
||||
def greet(): ... |
||||
``` |
||||
|
||||
```php |
||||
function greet() { ... } |
||||
|
||||
function main(): void |
||||
{ |
||||
global $greet; |
||||
$greet = b('greet'); |
||||
$greet = a('greet'); |
||||
... |
||||
} |
||||
``` |
||||
|
||||
- A decorator can be a defined function, a `from m import f` imported symbol, a module attribute, or a decorator factory (`@dec('x')` → `$greet = dec('x')('greet');`) |
||||
- The decorated function name is registered as a module global, and all call sites (including inside other function bodies) call the decorated result indirectly via `global` + variable: `$greet()` |
||||
- Recursive calls inside the decorated function body also resolve to the decorated variable, consistent with Python semantics |
||||
|
||||
## print / sys.exit degradation rules |
||||
|
||||
Degrade to native statements only when PHP behavior is fully identical to Python: |
||||
|
||||
| Form | Output | |
||||
|---|---| |
||||
| `print()` | `echo "\n";` | |
||||
| `print("a", "b")` (string/integer constants, module attributes, containers, f-strings) | `echo 'a', ' ', 'b', "\n";` | |
||||
| `print(1.5)`, `print(True)`, `print(x, sep=...)` | not degraded: `python\print(...)` | |
||||
| after user-defined/imported/assigned shadowing of `print` | not degraded | |
||||
| `sys.exit()` / `sys.exit(2)` (including the `from sys import exit` form) | `exit;` / `exit(2);` | |
||||
| `sys.exit("fail")` | not degraded: `sys\exit('fail');` | |
||||
|
||||
## Known behaviors (not errors, but worth noting) |
||||
|
||||
1. `import os.path` (no alias) only introduces the first segment `use python\os;`. |
||||
2. An explicit `global x` inside a function and the auto-injected `global x` for a module global appear twice (valid PHP). |
||||
3. Writing `print = str` (assigning a built-in name to a variable) treats the right side as a variable (`$print = $str;`), not as built-in name resolution. |
||||
4. Errors for bytes/complex literals have no line number (constants are encoded during the AST load stage; position information is not passed through). |
||||
5. Decorator rebinding uniformly happens at the start of `main()`, slightly differing from Python's exact "decorated at the def site" position; if a decorator expression depends on assignments later in the top-level statements, the evaluation timing may differ. |
||||
6. Decorated function names are registered as module globals, so the name appears in every function's auto-injected `global` list (redundant but valid). |
||||
|
||||
## Running tests |
||||
|
||||
```bash |
||||
vendor/bin/phpunit --filter 'PythonToTypePhpConverterTest|PythonAstLoaderTest|PythonToolsCommandTest' |
||||
``` |
||||
|
||||
Converter tests depend on a real `python3` to parse the AST, and are skipped automatically when the environment lacks it. |
||||
@ -0,0 +1,48 @@ |
||||
# TypePHP Compiler Internal Documentation |
||||
|
||||
This directory contains compiler implementation, compatibility, build-mode, and special-topic design documents. The user-facing manual lives in the separate `aot/docs` repository; the research reports and refactoring plans here may describe historical state, and current behavior should be determined by the code, tests, and compatibility checklist. |
||||
|
||||
## Current authoritative documents |
||||
|
||||
- [AOT and PHP Incompatible Features Checklist](INCOMPATIBLE_PHP_FEATURES.md): a concise list of current limitations. |
||||
- [Incompatibility Classification](PHP_INCOMPATIBILITY_CLASSIFICATION.md): distinguishes Hard Limit, Intentional Rule, Pending, and Partial. |
||||
- [Compiler CLI](COMPILER_CLI.md): current CLI arguments and project configuration. |
||||
- [Compilation Modes](COMPILATION_MODES.md): binary, extension, library modes. |
||||
- [Quick Start](QUICKSTART.md): the minimal compile flow. |
||||
- [Compile-time Functions](COMPILE_TIME_FUNCTIONS.md): `any()`, `refval()`, `objval()`, `expected()`, `unexpected()`, and keyword methods. |
||||
- [Native Types](NATIVE_TYPES.md), [High-Precision Types](HIGH_PRECISION_TYPES.md), [Std Containers](STD_CONTAINERS.md). |
||||
- [Three Object Storage and Passing Models](OBJECT_STORAGE_AND_PASSING_MODELS.md): the responsibilities, ABI, and non-substitutable boundaries of Zend Object, PHPX Box, and Native Class Object. |
||||
- [Universal and Extension Methods](UNIVERSAL_METHODS.md), [Generator](YIELD_GENERATOR.md). |
||||
- [`#[Immutable]` compile-time read-only contract](IMMUTABLE.md): methods, parameters, aliases, call boundaries, and dynamic escape rules. |
||||
- [`#[ArrayDef]` array property contract](ARRAY_DEF.md): List/Map metadata, direct-write checks, and dynamic escape boundaries. |
||||
- [Class Inheritance](CLASS_INHERITANCE.md), [Mixed C++/PHP](MIXED_CPP_PHP.md). |
||||
|
||||
## Architecture and maintenance |
||||
|
||||
- [Backend-Neutral IR](BACKEND_NEUTRAL_IR.md) |
||||
- [TypePHP WASM Technical Plan and Implementation Plan](TYPEPHP_WASM_IMPLEMENTATION_PLAN.md) |
||||
- [Building TypePHP WASI Programs](WASI_BUILD.md) |
||||
- [Rebuilding the PHPX WASM Static Library](PHPX_WASM_BUILD.md): incremental rebuild of `libphpx.a`, numeric-dependency rebuild, and full SDK rebuild boundaries. |
||||
- [Core Refactoring Plan](REFACTORING_PLAN.md) |
||||
- [Scope Management Design](SCOPE_MANAGEMENT.md): responsibilities and usage boundaries of `CallableScope`, `UserCodeScopeGuard`, and `FakeScopeGuard`. |
||||
- [Runtime Initialization and Shutdown Flow](RUNTIME_LIFECYCLE.html): the four-layer lifecycle of PHP, PHPX, TypePHP, and the project, covering bin/ext/lib, multi-module, and WASM. |
||||
- [C++ Namespaces, Prefixes, and Symbol ABI](CPP_SYMBOL_NAMING.md): responsibility boundaries and conflict rules for `typephp_`, `php::`, `typephp_<project>`, and user callables `php_`. |
||||
- [Zend Object Creation and Property Default Value Initialization](OBJECT_CREATION.md): the `gen_stub.php` default property table, trigger conditions for custom `create_object`, execution flow, and performance boundaries. |
||||
- [Native Class Object Design](NATIVE_CLASS_OBJECT.md) and [Implementation Acceptance Matrix](NATIVE_CLASS_IMPLEMENTATION_AUDIT.md). |
||||
- [PHP 8.4 Property Hook Integration Design](PROPERTY_HOOKS.md): compile-time lowering, Zend Hook metadata, object introspection, and PHPX ABI boundaries. |
||||
- [Interface Property Hook Implementation Plan](INTERFACE_PROPERTY_HOOKS.md): interface property contracts, compile-time variance checks, and PHP 8.4 abstract Hook metadata. |
||||
- [Build Speed Research](AOT_BUILD_SPEED_RESEARCH.md) |
||||
- [Optimization Priority](aot-optimization-priority.md) |
||||
- [High-Precision Type In-Place Operation Optimization Plan](BIG_NUMBER_INPLACE_OPTIMIZATION_PLAN.md) |
||||
- [GMP Differences](GMP_GAP.md) |
||||
|
||||
## Research and historical materials |
||||
|
||||
`hhvm-review.md`, `kphp-review.md`, `peachpie-review.md`, `phpstan-design-analysis.md`, `php-src-optimizer-analysis.md`, and the patent drafts record comparisons and design background at the time of investigation, and do not serve as the current feature list. |
||||
|
||||
## Maintenance rules |
||||
|
||||
1. When current compatibility changes, update `INCOMPATIBLE_PHP_FEATURES.md` and the classification document simultaneously. |
||||
2. All syntax and semantic limitations link uniformly to the current compatibility checklist to avoid maintaining duplicate lists. |
||||
3. Feature support should be determined by PHPT/PHPUnit regression tests. |
||||
4. Historical research documents preserve the original comparison conclusions and note the investigation date where necessary; they should not be silently rewritten to reflect the current state. |
||||
@ -0,0 +1,430 @@ |
||||
# AOT Compiler Core Refactoring Plan |
||||
|
||||
> For the next-phase OOA/OOD/OOP refactoring of `Translator`, `CompilerBase`, and `Preprocessor`, please use [CORE_OOA_OOD_OOP_REFACTORING_PLAN.md](CORE_OOA_OOD_OOP_REFACTORING_PLAN.md) as the implementation baseline. This document retains the historical plan of the earlier modularization refactoring. |
||||
|
||||
## Background |
||||
|
||||
The current core classes of the AOT compiler carry too many responsibilities. In particular, classes such as `CompilerBase` and `Translator` simultaneously contain AST dispatch, type inference, property access resolution, call resolution, code generation, diagnostics, and context state maintenance. As functionality continues to grow, this structure causes the following problems: |
||||
|
||||
- Insufficient encapsulation: modifying one semantic point easily affects multiple code paths. |
||||
- Insufficient code reuse: similar logic is repeatedly implemented across normal properties, static properties, nullsafe, assignment, isset/empty/refval, and other paths. |
||||
- Compile-time checks are prone to bypass paths: for example, some dynamic fallbacks do not reuse the static resolver. |
||||
- Individual classes are too large, and review, test localization, and long-term maintenance costs keep rising. |
||||
- Design boundaries are unclear: the type system, symbol resolution, property access, and call generation are too deeply coupled. |
||||
|
||||
This plan guides the subsequent incremental refactoring, aiming to improve architectural quality while avoiding the behavior regression risk of a large-scale one-shot rewrite. |
||||
|
||||
## Core Principles |
||||
|
||||
1. Incremental refactoring; a one-shot rewrite of the core compilation flow is forbidden. |
||||
2. Prioritize extracting pure logic with no or low state dependency, then logic that depends on the compilation context. |
||||
3. Each refactoring step should keep behavior unchanged as much as possible; behavior changes must be explained separately and covered by tests. |
||||
4. New modules should be modeled around a stable domain, not mechanically split by AST node. |
||||
5. Prefer small services/helpers, explicit DTOs, resolvers, and emitters; introduce patterns such as Visitor and Strategy only after boundaries stabilize. |
||||
6. All error messages remain PHP-style, do not expose implementation details, and do not use expressions like "AOT forbids". |
||||
7. Each phase must have phpunit or phpt regression verification, especially for high-risk paths such as properties, types, calls, inheritance, and exceptions. |
||||
8. Compiler interfaces exposed to resolver/emitter should preferably remain read-only. Read-only query interfaces can be made public as needed, but write operations must be handled with extra care to avoid bypassing unified state management. |
||||
|
||||
## Target Architecture Direction |
||||
|
||||
### CompilerBase |
||||
|
||||
It should ultimately converge into the compilation context, common utilities, AST dispatch entry point, and cross-module collaboration layer, no longer directly carrying a large amount of domain logic. |
||||
|
||||
Responsibilities to retain: |
||||
|
||||
- Current file, function, class, and method context management. |
||||
- Temporary variable, local variable, and scope state maintenance. |
||||
- Top-level AST dispatch entry point. |
||||
- Unified entry point for fatal/warning. |
||||
- Collaboration with lower-level resolver/emitter. |
||||
|
||||
Responsibilities to gradually move out: |
||||
|
||||
- Type declaration parsing and type compatibility determination. |
||||
- Property visibility, property offset, and typed property checks. |
||||
- Function and method call resolution. |
||||
- Complex expression code generation. |
||||
- Union/intersection/nullable runtime typecheck generation. |
||||
|
||||
### Translator |
||||
|
||||
Retain file-level, class-level, and function-level compilation flow control, and gradually reduce specific semantic checking and expression generation logic. |
||||
|
||||
Responsibilities to retain: |
||||
|
||||
- File scanning and translation entry point. |
||||
- Class, function, and method code block generation flow. |
||||
- class/function/constant/property metadata registration. |
||||
- Compilation artifact organization. |
||||
|
||||
Responsibilities to gradually move out: |
||||
|
||||
- Detail logic of inheritance compatibility checks. |
||||
- Trait property conflict check details. |
||||
- Parameter, return value, and property type compatibility determination. |
||||
|
||||
## Module Splitting Plan |
||||
|
||||
### 1. TypeSystem |
||||
|
||||
Responsibilities: |
||||
|
||||
- Type declaration parsing. |
||||
- Mapping from PHP types to AOT internal types. |
||||
- Nullable, union, and intersection type expansion. |
||||
- Type compatibility determination for parameters, return values, properties, and constants. |
||||
- Boundary definition between static types and runtime typecheck. |
||||
- Type string formatting. |
||||
|
||||
Recommended submodules: |
||||
|
||||
- `TypeResolver` |
||||
- `TypeCompatibility` |
||||
- `TypeCheckEmitter` |
||||
- `TypeStringFormatter` |
||||
|
||||
Design requirements: |
||||
|
||||
- union, intersection, and nullable can still be treated as mixed/any in the static phase, but runtime typecheck information must be preserved. |
||||
- The resolution rules for special type names such as `self`, `parent`, and `static` must be centralized to avoid inconsistent implementation across multiple paths. |
||||
- Type error messages must include necessary context such as function, method, parameter, and property. |
||||
|
||||
### 2. SymbolResolver |
||||
|
||||
Responsibilities: |
||||
|
||||
- Namespace resolution. |
||||
- use alias resolution. |
||||
- `self`, `parent`, `static` resolution. |
||||
- Class, interface, trait, function, and constant name normalization. |
||||
- Boundary determination between dynamic symbols and statically resolvable symbols. |
||||
|
||||
Recommended interfaces: |
||||
|
||||
- `resolveClassName(NodeAbstract $node): SymbolResolution` |
||||
- `resolveFunctionName(NodeAbstract $node): SymbolResolution` |
||||
- `resolveMethodScope(NodeAbstract $node): SymbolResolution` |
||||
- `resolveClassConstScope(NodeAbstract $node): SymbolResolution` |
||||
|
||||
Design requirements: |
||||
|
||||
- Different call paths must not reimplement the `self/parent/static` rules. |
||||
- For symbols that cannot be statically determined, explicitly return a dynamic state instead of silently degrading to string concatenation. |
||||
|
||||
### 3. PropertyAccessResolver |
||||
|
||||
Responsibilities: |
||||
|
||||
- Object property access resolution. |
||||
- Static property access resolution. |
||||
- Nullsafe property access static checking. |
||||
- private/protected/public visibility checking. |
||||
- Native property offset lookup. |
||||
- Entry point for typed property write typecheck information generation. |
||||
- Static-vs-instance property misuse checking. |
||||
|
||||
Recommended interfaces: |
||||
|
||||
- `resolveInstancePropertyAccess(PropertyAccessRequest $request): PropertyAccessResult` |
||||
- `resolveStaticPropertyAccess(StaticPropertyAccessRequest $request): PropertyAccessResult` |
||||
- `assertReadable(PropertyAccessResult $result): void` |
||||
- `assertWritable(PropertyAccessResult $result): void` |
||||
- `emitRead(PropertyAccessResult $result): string` |
||||
- `emitWrite(PropertyAccessResult $result, string $value): string` |
||||
|
||||
Paths that need unified coverage: |
||||
|
||||
- `$obj->prop` |
||||
- `$obj?->prop` |
||||
- `Class::$prop` |
||||
- `self::$prop` |
||||
- `parent::$prop` |
||||
- `static::$prop` |
||||
- `isset($obj->prop)` |
||||
- `empty($obj->prop)` |
||||
- `refval($obj->prop)` |
||||
- normal assignment, compound assignment, increment/decrement, unset. |
||||
|
||||
It is recommended to start from this module as the first priority, because recent problems are concentrated in property access and visibility bypass, and the test boundaries are relatively clear. |
||||
|
||||
### 4. CallResolver |
||||
|
||||
Responsibilities: |
||||
|
||||
- Function call resolution. |
||||
- Object method call resolution. |
||||
- Static method call resolution. |
||||
- Native call vs. dynamic call selection. |
||||
- Named args, unpack, and by-ref parameter handling. |
||||
- Closure and dynamic callable degradation rules. |
||||
|
||||
Recommended interfaces: |
||||
|
||||
- `resolveFunctionCall(CallRequest $request): CallResolution` |
||||
- `resolveMethodCall(MethodCallRequest $request): CallResolution` |
||||
- `resolveStaticCall(StaticCallRequest $request): CallResolution` |
||||
- `emitCall(CallResolution $resolution): string` |
||||
|
||||
Design requirements: |
||||
|
||||
- When the parameter information of a static function or built-in function is clear, references can be automatically converted. |
||||
- For dynamic calls, closures, and cases where by-ref parameter information cannot be obtained at compile time, an explicit `refval()` must be required. |
||||
- When using unpack with trailing named args appended, it should degrade to a dynamic call and must not go through a native call. |
||||
|
||||
### 5. ExpressionEmitter |
||||
|
||||
Responsibilities: |
||||
|
||||
- Expression-level code generation. |
||||
- Gradually split the large `parseExpr()` dispatch logic. |
||||
- Reuse resolver results to generate C++ code. |
||||
|
||||
It is recommended to split by domain rather than starting with a large number of AST visitors: |
||||
|
||||
- `AssignmentEmitter` |
||||
- `PropertyEmitter` |
||||
- `CallEmitter` |
||||
- `ArrayEmitter` |
||||
- `ControlExprEmitter` |
||||
- `ObjectEmitter` |
||||
|
||||
Design requirements: |
||||
|
||||
- Keep the existing `parseExpr()` as the dispatch entry point for now. |
||||
- Migrate only one group of expressions at a time, and run the corresponding test group after each migration. |
||||
- Handle expression side effects, evaluation order, and temporary variable generation conservatively. |
||||
|
||||
### 6. Diagnostic |
||||
|
||||
Responsibilities: |
||||
|
||||
- Unified fatal/warning construction. |
||||
- Provide context enhancement capability. |
||||
- Ensure error message style is close to PHP. |
||||
|
||||
Recommended capabilities: |
||||
|
||||
- Current function/method name. |
||||
- Parameter name. |
||||
- Property name and class name. |
||||
- Source location. |
||||
- Declaration location and usage location. |
||||
|
||||
Design requirements: |
||||
|
||||
- Error messages must not use "AOT" as the actor. |
||||
- For user-fixable problems, accurate symbol names should be included. |
||||
- Problems discoverable at compile time should preferably be compile-time fatal, and should not rely on runtime typecheck exceptions as a fallback. |
||||
|
||||
## Phase Plan |
||||
|
||||
### Phase 1: Property Access Resolution Modularization |
||||
|
||||
Objectives: |
||||
|
||||
- Extract property access resolution logic such as `findNativeProperty()`, `findNativeStaticProperty()`, `canAccessProtectedProperty()`. |
||||
- Keep generated code essentially unchanged. |
||||
- Establish a unified `PropertyAccessResult` carrying information such as property declaration, declaring class, accessing class, whether native, offset, and whether dynamic. |
||||
|
||||
Scope: |
||||
|
||||
- Normal object property reads. |
||||
- Static property reads. |
||||
- Nullsafe property static checking. |
||||
- Property visibility checking. |
||||
|
||||
Current progress: |
||||
|
||||
- Established `PropertyAccessResolver` and `PropertyAccessResult` as the first abstraction layer for property access resolution. |
||||
- Instance property reads now complete native property lookup, static-vs-instance checking, and visibility checking through the resolver's explicit `resolveNativeInstanceProperty()` interface. |
||||
- `findNativeStaticProperty()` now completes static property checking through the resolver's explicit `resolveNativeStaticProperty()` interface. |
||||
- Nullsafe property chain checking now completes class-name advancement and visibility checking through the resolver's explicit `resolveNullsafePropertyChain()` interface. |
||||
- The old generic `CompilerBase::findNativeProperty()` entry point has been removed to prevent further spread of access patterns carrying the `$static` boolean parameter. |
||||
- The old `findNativeStaticProperty(..., &$class)` by-ref protocol has been removed; static property reads now use explicit DTOs `StaticPropertyFetchTarget` and `StaticPropertyFetchResolution`. |
||||
- Target class resolution for instance property reads has been extracted to `InstancePropertyFetchTarget`; `getPropertyIdentifier()` no longer mixes target resolution, resolver calls, and dynamic fallback branches. |
||||
- The `nativeProperty`, `nativePropertyDef`, and `nativeClassDef` previously scattered on AST attributes have been merged into the `NativePropertyAccess` metadata to avoid inconsistent state among the three. |
||||
- Direct reads and writes of `nativePropertyVar`, `nativePropertyValueSource`, `objectProps`, and `staticPropRefs` have been converged into helper methods; business paths no longer determine property access semantics by string content. |
||||
- Typed instance property hoist and typed static property ref registration have been extracted into independent helpers, currently still keeping the original generated code structure. |
||||
- `CompilerBase::isSameClassName()`, `isSameOrSubclassOf()`, and `canAccessProtectedProperty()` have been delegated to the resolver to avoid further rule spread. |
||||
- Added `prepare/convert/idle` compilation phase states; `PropertyAccessResolver` can only be created and used in the convert phase to avoid misusing incomplete class table state in the preprocessing phase. |
||||
- `PropertyAccessResolver` has been changed to depend on the read-only `PropertyAccessContext` interface instead of fully depending on the large `CompilerBase` class. |
||||
- Established `PropertyAssignTypeInfo`, extracting the pure metadata computation for typed property writes, including fixed-type property determination, default values, the runtime typecheck list, and type strings. |
||||
- The current migration keeps generated code unchanged; read/write emitters will be unified in subsequent phases. |
||||
|
||||
Status: |
||||
|
||||
- Phase 1 is essentially wrapped up. Unless property read resolver bypass or behavior regression is found later, the scope of Phase 1 will not be further expanded. |
||||
- Phase 2 has begun; assignment, compound assignment, inc/dec, unset, and refval paths related to property writes still need to be further unified. |
||||
|
||||
Verification: |
||||
|
||||
- `phpunit/src/NativePropertyTest.php` |
||||
- `phpunit/src/InheritanceErrorTest.php` |
||||
- object property related phpt. |
||||
- static property related phpt. |
||||
- nullsafe related phpt. |
||||
|
||||
### Phase 2: Property Write Path Unification |
||||
|
||||
Objectives: |
||||
|
||||
- All property write paths resolve first, then emit. |
||||
- Typed property runtime typecheck converges from scattered logic into the property write module. |
||||
- Eliminate the problem where assignment, compound assignment, inc/dec, and unset each implement property access rules independently. |
||||
|
||||
Scope: |
||||
|
||||
- `$obj->prop = $value` |
||||
- `$obj->prop += $value` |
||||
- `$obj->prop++` |
||||
- `unset($obj->prop)` |
||||
- `Class::$prop = $value` |
||||
- `??=` related property paths. |
||||
|
||||
Verification: |
||||
|
||||
- typed property related phpunit/phpt. |
||||
- object property optimization related phpt. |
||||
- nullsafe write context error tests. |
||||
- private/protected/static property error tests. |
||||
|
||||
Current progress: |
||||
|
||||
- Phase 2 has begun. |
||||
- Established `PropertyWriteTarget` as the minimal target DTO for the property write path. |
||||
- Normal assignment and `??=` have been connected to `preparePropertyWriteTarget()`, uniformly completing property target preparation before writes, and executing static checks and runtime typecheck wrapping through `assertCanAssignPropertyWrite()` and `wrapPropertyWriteTypeCheck()`. |
||||
- `getProperty()` / `setProperty()` generation for dynamic object properties has been converged into the `emitDynamicPropertyRead()` / `emitDynamicPropertyWrite()` helpers; normal dynamic property assignment, compound assignment, and increment/decrement now reuse this entry point. |
||||
- The dynamic property path of compound assignment has been connected to `preparePropertyWriteTarget()`, uniformly completing property write target preparation and static checks first. |
||||
- `PropertyWriteTarget` has begun carrying the object/property expressions of safe dynamic property write targets; normal dynamic property assignment, compound assignment, and increment/decrement now prefer emitting code through target-level read/write helpers. |
||||
- Dynamic property `unset`, property array dimension writes, and safe object property reference paths in reference arguments/refval/reference assignment have begun reusing target-level unset/ref helpers. |
||||
- Target/ref generation for object property reference expressions has been converged into `emitDynamicPropertyFetchRef()`; the unused old static property assignment entry point has been deleted, and static property assignment continues through the unified assignment target path. |
||||
- The dynamic object/property fields of `PropertyWriteTarget` have been encapsulated as getters; property array dimension writes have been connected to target-level append/update emitters. |
||||
- Established the `emitDynamicPropertyFetchRead/Write/Unset/AppendArray/UpdateArray()` wrapper layer; callers only pass in the property access AST and an optional target, and `CompilerBase` uniformly selects the target path or the old fallback path. |
||||
- Normal assignment, compound assignment, increment/decrement, unset, property array dimension writes, and reference assignment have removed the direct branch determination of dynamic targets in the Parser trait, instead reusing the unified emitter wrappers. |
||||
- To avoid changing the evaluation order of complex expressions, currently only dynamic property writes whose object part is a variable have their target object/property fields populated; complex object expressions still retain the old path. |
||||
- The current step keeps generation logic compatible for valid code, but will route more property write paths into unified static checking; continue converging static/native property write emitters and `??=` property write result generation. |
||||
|
||||
### Phase 3: Type System Modularization |
||||
|
||||
Objectives: |
||||
|
||||
- Extract type declaration parsing, type compatibility, and runtime typecheck generation. |
||||
- Clarify the responsibility boundary between static type inference and runtime typecheck. |
||||
- Reduce the repeated handling of type rules across parameters, return values, properties, and constants. |
||||
|
||||
Scope: |
||||
|
||||
- `parseTypeDecl()`. |
||||
- `buildTypeCheckFromNode()`. |
||||
- Parameter typecheck. |
||||
- Return value typecheck. |
||||
- Property typecheck. |
||||
- Constant type handling. |
||||
|
||||
Verification: |
||||
|
||||
- union, intersection, nullable type tests. |
||||
- Parameter, return value, property typecheck tests. |
||||
- namespace constant tests. |
||||
- constructor, void expression related tests. |
||||
|
||||
### Phase 4: Call Resolution Modularization |
||||
|
||||
Objectives: |
||||
|
||||
- Unify function, method, and static method call resolution. |
||||
- Clarify degradation rules for native calls, dynamic calls, and closure calls. |
||||
- Centrally handle named args, unpack, and by-ref parameters. |
||||
|
||||
Scope: |
||||
|
||||
- `parseFuncCall()`. |
||||
- `parseMethodCall()`. |
||||
- `parseStaticCall()`. |
||||
- call args parsing. |
||||
- native/internal/user function call paths. |
||||
|
||||
Verification: |
||||
|
||||
- named args, unpack related phpt. |
||||
- by-ref parameter related phpt. |
||||
- closure related phpt. |
||||
- parent/self/static call related phpt. |
||||
|
||||
### Phase 5: Expression Generator Splitting |
||||
|
||||
Objectives: |
||||
|
||||
- Migrate the large amount of expression generation logic behind `parseExpr()` to domain emitters. |
||||
- `CompilerBase` retains dispatch and shared context capabilities. |
||||
- Reduce the code size of individual files and classes. |
||||
|
||||
Scope: |
||||
|
||||
- AssignmentEmitter. |
||||
- PropertyEmitter. |
||||
- CallEmitter. |
||||
- ArrayEmitter. |
||||
- ControlExprEmitter. |
||||
|
||||
Verification: |
||||
|
||||
- Run the corresponding test group after each emitter migration. |
||||
- Finally run the core phpunit and selected phpt regression set. |
||||
|
||||
### Phase 6: Translator Convergence |
||||
|
||||
Objectives: |
||||
|
||||
- Extract inheritance compatibility, trait merging, and class member validation logic into dedicated checkers. |
||||
- `Translator` focuses on compilation flow organization. |
||||
|
||||
Recommended submodules: |
||||
|
||||
- `InheritanceChecker` |
||||
- `TraitCompositionChecker` |
||||
- `ClassMemberValidator` |
||||
- `FunctionSignatureChecker` |
||||
|
||||
Verification: |
||||
|
||||
- inheritance error phpunit. |
||||
- trait related phpt. |
||||
- interface/abstract/final/readonly related phpt. |
||||
|
||||
## Test Gate |
||||
|
||||
Each refactoring PR or phase must at least satisfy: |
||||
|
||||
- Relevant phpunit must pass. |
||||
- Relevant phpt must pass. |
||||
- If C++/phpx is modified, gtest must be supplemented and the corresponding tests must pass. |
||||
- If a new compile-time error is introduced, a fixed fixture must be added to `phpunit/code` or a new phpt must be added. |
||||
- Do not use `file_put_contents()` to temporarily generate source code as a new test method. |
||||
|
||||
It is recommended to maintain a minimal regression set per module: |
||||
|
||||
- Property module: `NativePropertyTest`, `InheritanceErrorTest`, object property, static property, nullsafe. |
||||
- Type module: type_decl, type_hits, typed property, union/intersection/nullable. |
||||
- Call module: function call, method call, parent_call, closure, named args, unpack, by-ref. |
||||
- Control flow and expressions: ternary, match, goto, loop, array, coalesce, void expression. |
||||
|
||||
## Risk Control |
||||
|
||||
- Do not perform large-scale file movement and behavior changes in the same change. |
||||
- Supplement tests for current behavior before each migration, especially the paths corresponding to historical bugs. |
||||
- Keep old entry points for a period, calling new modules through adapters to reduce switching risk. |
||||
- Remain conservative about dynamic PHP semantics; do not over-optimize when it cannot be statically determined. |
||||
- For places where AOT is clearly incompatible with PHP's historical baggage, express them as language rules in documentation and error messages, not as implementation limitations. |
||||
|
||||
## Recommended Next Step |
||||
|
||||
Start from `PropertyAccessResolver`. |
||||
|
||||
Reasons: |
||||
|
||||
- Recent bugs are mostly concentrated in property access, visibility, typed property, nullsafe, and static-vs-instance paths. |
||||
- Existing tests are relatively easy to extend. |
||||
- Property access is a relatively independent domain outside the type system, optimizer, and call generation, making it suitable for establishing the resolver/result pattern first. |
||||
- After completion, it can directly reduce the complex branches in `CompilerBase` and lay the foundation for subsequent ExpressionEmitter splitting. |
||||
@ -0,0 +1,401 @@ |
||||
# TypePHP Scope Management Design |
||||
|
||||
This document is an internal implementation document for TypePHP and PHPX. It explains the responsibilities, implementation, lifecycle, performance characteristics, and applicable scenarios of the three current scope managers. The "scope" here is not a single Zend concept: callable resolution, execution-frame class scope, and `EG(fake_scope)` each serve different subsystems and cannot be substituted for one another. |
||||
|
||||
## 1. Design Goals |
||||
|
||||
The C++ methods generated by TypePHP are not ordinary Zend user functions. When a dynamic call returns to the ZendVM, Zend still needs the following information to reproduce PHP's visibility rules: |
||||
|
||||
- The lexical scope of the declared method, used to determine whether private/protected members are accessible; |
||||
- The called scope of the current late static binding; |
||||
- The current instance `$this`, used to resolve non-static method callables; |
||||
- The `EG(fake_scope)` read by certain Zend property, object, and exception APIs. |
||||
|
||||
The Scope design follows these principles: |
||||
|
||||
1. Prefer passing scope explicitly, and do not modify Zend's global or real execution-frame state. |
||||
2. Create a reusable callable context at most once per AOT method call; multiple calls within a loop share it. |
||||
3. Temporarily modify the nearest user-code frame only when the compiler cannot determine the callback location. |
||||
4. Use RAII when modifying Zend executor state, and guarantee restoration on the exception path. |
||||
5. Do not pay extra wrapping cost for pure Native Calls or public, absolutely-located callbacks. |
||||
|
||||
## 2. Overview |
||||
|
||||
| Manager | Managed State | Primary Purpose | Modifies Zend Current State | |
||||
| --- | --- | --- | --- | |
||||
| `php::CallableScope` | A synthetic `zend_execute_data` containing lexical scope, called scope, and `$this` | Dynamic method calls, first-class callables, built-in function callbacks | No | |
||||
| `php::UserCodeScopeGuard` | The `zend_function::common.scope` of the nearest user-code frame | `call_user_func*` and dynamic call paths where the callback is hidden inside argument unpacking | Yes, restored on destruction | |
||||
| `php::FakeScopeGuard` | `EG(fake_scope)` | Zend property, object, exception, and other APIs that read fake scope | Yes, restored on destruction or explicit `restore()` | |
||||
|
||||
The selection rule can be simplified as: |
||||
|
||||
- A concrete callable value is available: use `CallableScope`. |
||||
- Calling `call_user_func*`, or the callback of another built-in function is hidden in `...$args`: use `UserCodeScopeGuard`. |
||||
- The called Zend API explicitly reads `EG(fake_scope)`: use `FakeScopeGuard`. |
||||
- Pure native calls or operations that do not depend on caller visibility: do not create any Scope manager. |
||||
|
||||
## 3. `php::CallableScope` |
||||
|
||||
### 3.1 Responsibilities |
||||
|
||||
`CallableScope` is the main path for ordinary callable resolution today. It explicitly hands the caller context to `zend_is_callable_at_frame()` to: |
||||
|
||||
- Resolve private/protected methods; |
||||
- Resolve `self`, `parent`, and `static` callbacks; |
||||
- Preserve the called scope of late static binding; |
||||
- Provide the real `$this` for non-static methods; |
||||
- Invoke dynamic methods without modifying `EG(current_execute_data)` or the real execution frame. |
||||
|
||||
It does not handle property access and does not set `EG(fake_scope)`. |
||||
|
||||
### 3.2 Internal Structure |
||||
|
||||
The class is defined in PHPX's `include/phpx.h` and holds: |
||||
|
||||
```cpp |
||||
zend_function *caller_function_; |
||||
zend_class_entry *called_scope_; |
||||
zend_object *this_object_; |
||||
mutable zend_execute_data frame_{}; |
||||
``` |
||||
|
||||
On construction, it initializes a synthetic frame through `zend_vm_init_call_frame()`: |
||||
|
||||
- `caller_function_->common.scope` is the lexical scope, i.e. the class that declares the current method; |
||||
- `called_scope_` is the runtime called scope; |
||||
- Instance calls set `ZEND_CALL_HAS_THIS` and carry the real `zend_object *`; |
||||
- Static calls do not carry an object and only pass the called scope; |
||||
- If the called scope is empty, it falls back to the lexical scope. |
||||
|
||||
Resolution calls: |
||||
|
||||
```cpp |
||||
zend_is_callable_at_frame(callable, object, &frame_, 0, cache, error); |
||||
``` |
||||
|
||||
The synthetic frame is not installed into `EG(current_execute_data)`, so it does not pollute the current Zend call stack, and global state does not need to be restored on exit. |
||||
|
||||
### 3.3 Lifecycle and Ownership |
||||
|
||||
`CallableScope` does not own `zend_function`, `zend_class_entry`, or `zend_object`; it only borrows these pointers within the current AOT method stack frame: |
||||
|
||||
- TypePHP-compiled methods use persistent `zend_function`, whose lifecycle spans the request invocation; |
||||
- A Closure's `zend_function *` is valid for the lifetime of the Closure object; |
||||
- `$this` is valid during execution of the current method; |
||||
- `CallableScope` is non-copyable and non-movable, preventing the synthetic frame from being accidentally transferred or stored across lifecycles. |
||||
|
||||
`CallableScope` must not be cached beyond the request, nor be allowed to outlive its owning method or Closure. |
||||
|
||||
### 3.4 Compiler Generation Pattern |
||||
|
||||
The compiler lazily requests a Scope variable through `FunctionContext::$callableScopeVar`. The first time an explicit callable scope is needed, `getCallableScopeExpr()` allocates a temporary variable; subsequently `genScopeVarDecl()` hoists the initialization code to the function entry: |
||||
|
||||
```cpp |
||||
php::CallableScope tmp_var_1 = php::getCallableScope( |
||||
get_persistent_method(...), |
||||
this_ |
||||
); |
||||
``` |
||||
|
||||
`php::getCallableScope()` builds both the called scope and the real instance information from `this_`. All scoped calls within a method reference the same `tmp_var_1`, so repeated calls within a loop do not recreate the synthetic frame. |
||||
|
||||
If a method never uses scoped dynamic calls, first-class callables, or scoped callbacks, the compiler does not generate this variable. |
||||
|
||||
### 3.5 Usage Entry Points |
||||
|
||||
#### `php::callScoped()` |
||||
|
||||
Used for dynamic function or object method calls. Internally, `call_function_impl()` uses `CallableScope::resolve()` to obtain a `zend_fcall_info_cache`, then executes `zend_call_function()`. |
||||
|
||||
The typical scenario is when the compiler cannot resolve an object method into a Native Call, but still needs to preserve access to the current class's private/protected members. |
||||
|
||||
#### `php::makeScopedCallable()` |
||||
|
||||
Used for first-class callable syntax. The result of this syntax must be a real `Closure`, so even if the target method is public, it cannot simply return the original callback array or string. |
||||
|
||||
```php |
||||
$callback = self::privateMethod(...); |
||||
$callback = $this->publicMethod(...); |
||||
``` |
||||
|
||||
Ordinary methods create a Closure through `zend_create_fake_closure()`. If Zend returns `ZEND_ACC_CALL_VIA_TRAMPOLINE`, a forwarding Closure is used to preserve the dynamic semantics of magic `__call()` / `__callStatic()`. |
||||
|
||||
#### `php::prepareScopedCallback()` |
||||
|
||||
Used to pass a callback to PHP built-in functions such as `array_map()` and `usort()`. The goal here is only for the built-in function to invoke the callback correctly; it is not required that the argument itself become a Closure. |
||||
|
||||
Therefore, it first reuses the original value of the following callbacks: |
||||
|
||||
- Public methods; |
||||
- Located by absolute class name; |
||||
- Not relying on a trampoline. |
||||
|
||||
Only private/protected methods, `self` / `parent` / `static` relative callbacks, or trampolines create a Closure. This avoids unconditionally allocating a fake Closure each time a built-in function is called within a loop. |
||||
|
||||
### 3.6 Why `self` / `parent` / `static` Still Need Runtime Recognition |
||||
|
||||
In direct syntax, `self::class` can be expanded to a concrete class name at compile time, but PHP callbacks also allow dynamic values: |
||||
|
||||
```php |
||||
$class = 'self'; |
||||
$callback = [$class, 'method']; |
||||
``` |
||||
|
||||
In this case, only at runtime can we know whether the class name in the array is a relative class name. Therefore `isRelativeCallableClass()` cannot be fully moved to compile time. For known absolute public callbacks, this check returns false quickly and reuses the original value. |
||||
|
||||
## 4. `php::UserCodeScopeGuard` |
||||
|
||||
### 4.1 Responsibilities and Scope of Application |
||||
|
||||
`UserCodeScopeGuard` serves fully dynamic `call_user_func()` / `call_user_func_array()`, callback maps, and argument-unpacking scenarios where the compiler cannot statically rewrite the callback. |
||||
|
||||
```php |
||||
$args = [[$this, 'privateMethod'], 1]; |
||||
call_user_func(...$args); |
||||
``` |
||||
|
||||
A built-in function callback may be at a fixed position, a reverse position, in a named argument, or even a single function may have multiple callbacks. Before executing the `...$args` unpacking, the compiler does not know the final positional/named argument layout and cannot call `prepareScopedCallback()` only on the corresponding values. |
||||
|
||||
`call_user_func*` itself is a fully dynamic call boundary of the ZendVM; regardless of whether the callback appears explicitly, no fake Closure is created. If the callable array uses `self`, `parent`, or `static`, `normalizeCallableClass()` first converts the class part into a real class name: |
||||
|
||||
- `self` becomes `CallableScope::lexicalScope()`; |
||||
- `parent` becomes the parent class of the lexical scope; |
||||
- `static` becomes `CallableScope::calledScope()`. |
||||
|
||||
Normalization only copies the callback arrays that need modification. Absolute class names, object callbacks, Closures, and ordinary function names keep their original values. |
||||
|
||||
`preg_replace_callback_array()` is a special case of a callback map. Zend resolves callbacks in the map item by item internally; if the map were wrapped in advance, each call would perform an O(N) scan and might trigger array COW and multiple Closure allocations. Therefore the compiler keeps the original map and creates a `UserCodeScopeGuard` once at the method entry, letting Zend resolve directly with the correct scope. |
||||
|
||||
Apart from fully dynamic calls, callback maps, and argument unpacking, ordinary callback arguments must not use this guard; as long as the AST parameter position of a single callback is known, the `CallableScope` path should be used. |
||||
|
||||
### 4.2 Implementation |
||||
|
||||
The constructor walks upward from `EG(current_execute_data)` to find the nearest user-code frame, skipping internal frames: |
||||
|
||||
```cpp |
||||
while (frame && (!frame->func || !ZEND_USER_CODE(frame->func->type))) { |
||||
frame = frame->prev_execute_data; |
||||
} |
||||
``` |
||||
|
||||
Once found, it saves it and uses `CallableScope::lexicalScope()` to set the visibility scope: |
||||
|
||||
```cpp |
||||
function_ = frame->func; |
||||
previous_scope_ = function_->common.scope; |
||||
function_->common.scope = callable_scope.lexicalScope(); |
||||
``` |
||||
|
||||
The destructor restores `previous_scope_`. The class is non-copyable and non-movable, ensuring one construction corresponds to one restoration. If no user-code frame is available, it throws: |
||||
|
||||
```text |
||||
A user-code frame is required for scoped dynamic callback calls |
||||
``` |
||||
|
||||
This guard operates on the user-code frame found from the current request's execution chain, not on the persistent internal methods registered by TypePHP in MINIT. `EG(current_execute_data)` itself belongs to the current executor context. Its impact window is limited to the RAII lifecycle of the current AOT method call. |
||||
|
||||
### 4.3 Compiler Generation Pattern |
||||
|
||||
The compiler maintains a semantically clear flag: |
||||
|
||||
```php |
||||
FunctionContext::$needsUserCodeCallableScope |
||||
``` |
||||
|
||||
When the compiler encounters a dynamic callback of `call_user_func*`, or a built-in function known to synchronously invoke callbacks has an argument unpacking that cannot be matched, `markUserCodeCallableScope()` sets this flag. The state belongs to the current `FunctionContext`, so ordinary methods, nested Closures, and Fibers are independent and do not leak the guard into an outer function incorrectly. Each function body generates only one at its entry: |
||||
|
||||
```cpp |
||||
php::CallableScope tmp_var_1 = php::getCallableScope(..., this_); |
||||
php::UserCodeScopeGuard tmp_var_2{tmp_var_1}; |
||||
``` |
||||
|
||||
Even if the call form is `call_user_func($closure)`, and inside the Closure there is another call via |
||||
`call_user_func(['self', 'method'])`, each layer only reads its own |
||||
`FunctionContext`, lexical scope, and `$this`, and cannot reuse or pollute the outer guard. |
||||
|
||||
It is not created per call site or per loop iteration. Methods without the above dynamic callbacks incur no such cost. |
||||
|
||||
### 4.4 Why This Fallback Is Currently Kept |
||||
|
||||
If it were completely removed, the compiler would have to add a structured argument binding and rewriting flow after argument unpacking completes, correctly handling: |
||||
|
||||
- Merging positional and named arguments; |
||||
- Forward and reverse positions of callbacks; |
||||
- Multiple callbacks in one function; |
||||
- Callback maps; |
||||
- PHP error semantics when arguments are duplicated, missing, or overridden during unpacking. |
||||
|
||||
This is not a localized replacement, but a medium-scale refactoring of `parseCallArgs()` and the argument container generation flow. Until a unified runtime argument post-processing mechanism is completed, keeping the strictly controlled `UserCodeScopeGuard` is simpler and more reliable. |
||||
|
||||
## 5. `php::FakeScopeGuard` |
||||
|
||||
### 5.1 Responsibilities |
||||
|
||||
`FakeScopeGuard` is the RAII wrapper for `EG(fake_scope)`. Some Zend APIs do not accept an explicit call frame; instead, they directly read `EG(fake_scope)` to determine class member visibility or perform class-scope-related operations. Only these APIs should use it. |
||||
|
||||
Current typical scenarios include: |
||||
|
||||
- Dynamic property reads, writes, and property hooks; |
||||
- Zend object handler calls; |
||||
- Default values or object initialization under class scope; |
||||
- Zend operations related to exception objects; |
||||
- Other Zend internal interfaces that explicitly read `EG(fake_scope)`. |
||||
|
||||
TypePHP's property access generator passes the current fake scope to the PHPX property helper through `FakeScopeGuard::current()`. |
||||
|
||||
### 5.2 Implementation |
||||
|
||||
On construction it saves the old value and sets the new value; on destruction it restores it: |
||||
|
||||
```cpp |
||||
explicit FakeScopeGuard(Scope scope) noexcept : previous_(current()) { |
||||
EG(fake_scope) = scope; |
||||
} |
||||
|
||||
~FakeScopeGuard() noexcept { |
||||
restore(); |
||||
} |
||||
``` |
||||
|
||||
`Scope` is deduced through `decltype(EG(fake_scope))` to be compatible with both PHP 8.4's mutable pointer and PHP 8.5's pointer-to-const. `restore()` is idempotent and can be safely called once in advance. |
||||
|
||||
### 5.3 Zend Bailout Considerations |
||||
|
||||
C++ exception unwinding executes destructors, but Zend bailout uses `longjmp` and does not execute C++ destructors. If a guard's lifecycle crosses a bailout boundary, it must be explicitly invoked in the corresponding `zend_catch` path: |
||||
|
||||
```cpp |
||||
fake_scope_guard.restore(); |
||||
``` |
||||
|
||||
and then continue the bailout or convert the exception. Relying solely on the destructor to handle bailout is incorrect. |
||||
|
||||
### 5.4 Non-applicable Scenarios |
||||
|
||||
`FakeScopeGuard` cannot replace `CallableScope`: |
||||
|
||||
- It has no synthetic frame; |
||||
- It cannot carry `$this`; |
||||
- It cannot fully express lexical scope and called scope; |
||||
- The resolution semantics of `zend_is_callable_at_frame()` should not be indirectly simulated through a global fake scope. |
||||
|
||||
Likewise, `EG(fake_scope)` must not be unconditionally set at the entry of every AOT method just because "private access might be needed." This would widen the impact of global state and make unrelated native-intensive calls bear the cost. |
||||
|
||||
## 6. Call Flows of the Three Scopes |
||||
|
||||
### 6.1 Known Dynamic Method Call |
||||
|
||||
```text |
||||
AOT method entry |
||||
-> lazily generated CallableScope |
||||
-> php::callScoped() |
||||
-> CallableScope::resolve() |
||||
-> zend_is_callable_at_frame(synthetic frame) |
||||
-> zend_call_function() |
||||
``` |
||||
|
||||
The whole process does not modify the real Zend frame. |
||||
|
||||
### 6.2 Known Built-in Function Callback |
||||
|
||||
```text |
||||
compiler marks callback argument |
||||
-> prepareScopedCallback(value, CallableScope) |
||||
-> public absolute callback: reuse value |
||||
-> scoped/trampoline callback: create Closure |
||||
-> call PHP internal function |
||||
``` |
||||
|
||||
First-class callables use the same resolution basis but must call `makeScopedCallable()` and return a Closure. |
||||
|
||||
### 6.3 Callback Inside Argument Unpacking |
||||
|
||||
```text |
||||
AOT method entry |
||||
-> UserCodeScopeGuard changes nearest user-code frame scope |
||||
-> internal function receives expanded arguments |
||||
-> Zend resolves hidden callback using that frame scope |
||||
-> method exit / C++ exception unwind |
||||
-> guard restores original scope |
||||
``` |
||||
|
||||
### 6.4 Property or Object Handler |
||||
|
||||
```text |
||||
save EG(fake_scope) |
||||
-> install FakeScopeGuard |
||||
-> call Zend property/object API |
||||
-> restore on normal/C++ exception exit |
||||
-> explicitly restore in zend_catch if bailout is possible |
||||
``` |
||||
|
||||
## 7. Forbidden Mixing and Maintenance Constraints |
||||
|
||||
1. Do not use `FakeScopeGuard` to resolve callables. |
||||
2. Do not modify the real user-code frame for ordinary known callbacks; use `prepareScopedCallback()`. |
||||
3. Do not let `UserCodeScopeGuard` become the general entry point for all dynamic calls again. |
||||
4. Do not recreate `CallableScope` at call sites within loops; it should be hoisted to the method entry by `FunctionContext` and reused. |
||||
5. Do not cache the function, object, or synthetic frame borrowed by `CallableScope` beyond the request. |
||||
6. Do not change first-class callables to return the original callback; its PHP result type must be Closure. |
||||
7. When adding a PHP built-in function that synchronously invokes callbacks, update the callback argument description table, noting the position, argument name, and whether it is a callback map. |
||||
8. Functions that save a callback but do not invoke it immediately must not mark the scope fallback merely because they receive a callable, for example `spl_autoload_register()`. |
||||
9. When adding a `FakeScopeGuard` usage that crosses a Zend bailout, code review must check whether `zend_catch` explicitly restores it. |
||||
|
||||
## 8. Performance Model |
||||
|
||||
| Path | Main Cost | Optimization Strategy | |
||||
| --- | --- | --- | |
||||
| `CallableScope` | Initializing one synthetic frame | At most once per AOT method, reused across loops | |
||||
| `callScoped()` | Dynamic resolution by `zend_is_callable_at_frame()` | Used only for dynamic calls; resolvable Native Calls do not enter this path | |
||||
| `prepareScopedCallback()` | One callable resolution | Public absolute callbacks do not create a Closure | |
||||
| `makeScopedCallable()` | Callable resolution and Closure allocation | Used only for first-class callables | |
||||
| `UserCodeScopeGuard` | One pointer lookup and write at method entry, plus restoration at exit | Generated only for `call_user_func*`, callback maps, or unresolved unpack callbacks | |
||||
| `FakeScopeGuard` | Two executor-global pointer assignments | Only surrounds Zend APIs that actually read fake scope | |
||||
|
||||
This design deliberately keeps common pure Native Calls, callback-free methods, and public callbacks on the shortest path. Do not sink low-frequency fallbacks into every call just to unify the surface form. |
||||
|
||||
## 9. Test Requirements |
||||
|
||||
Scope changes should cover at least the following layers: |
||||
|
||||
- PHPX unit tests: `FakeScopeGuard` save, nesting, restoration, and early `restore()`; |
||||
- Compiler structure tests: one method generates only one `php::getCallableScope()`, with multiple call sites reusing the same variable; |
||||
- PHPT: private/protected callbacks, non-static `self::method(...)`, public callbacks; |
||||
- PHPT: a mix of public and scoped callbacks in a callback map; |
||||
- PHPT: a private callback inside `...$args` is callable, and the scope is restored after an exception exit; |
||||
- PHPT: scope generation paths in Closure, Fiber, and ordinary methods; |
||||
- Regression tests: pure Native Calls must not generate extra Scope guards. |
||||
|
||||
Current related tests include: |
||||
|
||||
- `phpunit/src/ScopedCallContextTest.php` |
||||
- `phpunit/code/scoped-call-context-reuse.php` |
||||
- `tests/compiler/place-holder/non-static-self.phpt` |
||||
- `tests/compiler/callable/scoped-internal-callbacks.phpt` |
||||
- `tests/compiler/callable/unpacked-callback-scope-restored.phpt` |
||||
- PHPX `tests/src/scope_guard.cpp` |
||||
|
||||
PHPT tests involving dynamic calls that throw exceptions may trigger known ZendVM memory leak reports; only when the leak is confirmed to come from Zend's dynamic call exception path may the test locally set `USE_ZEND_ALLOC=0`, and memory checking must not be disabled globally. |
||||
|
||||
## 10. Code Location Index |
||||
|
||||
| Content | Location | |
||||
| --- | --- | |
||||
| `CallableScope` and public helper declarations | `vendor/swoole/phpx/include/phpx.h` | |
||||
| Callable resolution and wrapping | `vendor/swoole/phpx/src/core/base.cc`, `vendor/swoole/phpx/src/core/closure.cc` | |
||||
| `FakeScopeGuard` | `vendor/swoole/phpx/include/phpx_fake_scope_guard.h` | |
||||
| `UserCodeScopeGuard` | `vendor/swoole/phpx/include/typephp_helper.h`, `src/core/scope.cc` | |
||||
| `php::getCallableScope()` | `vendor/swoole/phpx/include/typephp_helper.h` | |
||||
| Callback marking and Scope variable generation | `src/CompilerBase.php` | |
||||
| Callback argument wrapping | `src/Generator/CallArgumentGenerator.php` | |
||||
| Closure/Fiber fallback guard | `src/Generator/ClosureGenerator.php`, `FiberGenerator.php` | |
||||
| Method fallback guard | `src/Translator.php` | |
||||
| Scope state | `src/Context/FunctionContext.php` | |
||||
| Fake scope in property access | `src/Parser/PropertyAccessTrait.php` | |
||||
|
||||
## 11. Future Evolution Principles |
||||
|
||||
`UserCodeScopeGuard` is a long-term retained mechanism for complex dynamic calls and is not targeted for removal. It modifies the user-code frame in the current thread and current request, and restores it via RAII; under ZTS, different threads have their own execution contexts, so the modified frame state is not shared. |
||||
|
||||
`CallableScope` is used for the single scenario where the compiler can determine the callback location and call boundary, in order to reduce frame modification and Closure wrapping; it is a faster, more explicit path, not one required to cover all scenarios such as unpacking and multi-layer dynamic callbacks. When encountering combinations that are hard to prove safe statically, prefer keeping `UserCodeScopeGuard`, and do not forcibly rewrite it to `CallableScope` for the sake of formal uniformity. |
||||
|
||||
Before adding a new Scope abstraction in the future, first confirm whether the Zend API depends on a synthetic call frame, a real user-code frame, or `EG(fake_scope)`. The name and type should directly express the managed Zend state, avoiding the reappearance of an overly broad general-purpose `Scope` class. |
||||
@ -0,0 +1,607 @@ |
||||
# Swoole AOT Strongly-Typed High-Performance Containers — Array Access Performance Improved by 10x |
||||
|
||||
> Std Container uses a PHPX Box to hold a concrete C++ template instance. For its storage |
||||
> and passing boundary relative to ordinary Zend Objects and Native |
||||
> Class Objects, see |
||||
> [OBJECT_STORAGE_AND_PASSING_MODELS.md](OBJECT_STORAGE_AND_PASSING_MODELS.md). |
||||
|
||||
The Swoole AOT compiler provides PHP with a set of `std` strongly-typed containers for replacing PHP Arrays in some performance-sensitive paths under AOT compile scenarios. They keep access syntax close to PHP while letting the compiler obtain a definite element type, key type, and container structure, thereby generating more direct, lower-overhead C++ code. |
||||
|
||||
## The Problem with PHP Arrays |
||||
|
||||
PHP Array is a very flexible data structure that can serve as a list, a hash table, a dictionary, or a struct: |
||||
|
||||
```php |
||||
$data = []; |
||||
$data[] = 1; |
||||
$data["name"] = "swoole"; |
||||
$data[10] = new stdClass(); |
||||
``` |
||||
|
||||
This flexibility brings convenience, but it also causes problems in large-scale projects and high-performance scenarios. |
||||
|
||||
### Programming-Convention Problems |
||||
|
||||
The key and value types of a PHP Array are not fixed, which easily leads to implicit conventions: |
||||
|
||||
```php |
||||
$user = [ |
||||
"id" => 1, |
||||
"name" => "alice", |
||||
"tags" => ["php", "swoole"], |
||||
]; |
||||
``` |
||||
|
||||
Structures like this usually rely on comments, documentation, or team conventions to guarantee correctness: |
||||
|
||||
```php |
||||
/** |
||||
* @param array{id:int, name:string, tags:string[]} $user |
||||
*/ |
||||
function saveUser(array $user): void |
||||
{ |
||||
} |
||||
``` |
||||
|
||||
But the runtime does not naturally guarantee: |
||||
|
||||
- `id` is always an int |
||||
- `name` always exists |
||||
- `tags` is always an array of strings |
||||
- whether the array is contiguous |
||||
- whether the key is int or string |
||||
- whether values of other types are mixed in |
||||
|
||||
This leads to a large amount of defensive code: |
||||
|
||||
```php |
||||
if (!isset($user["id"]) || !is_int($user["id"])) { |
||||
throw new InvalidArgumentException("invalid user id"); |
||||
} |
||||
``` |
||||
|
||||
In AOT compile scenarios, uncertain types also limit compiler optimization. When the compiler cannot reliably infer the element types inside an array, it can only conservatively generate generic `php::Array` / `php::Var` operations. |
||||
|
||||
### Performance Problems |
||||
|
||||
PHP Array is a generic HashTable suited to dynamic-language semantics, but it is not the optimal data structure for all scenarios. |
||||
|
||||
Typical overheads include: |
||||
|
||||
- each element needs to store zval type information |
||||
- key/value are both dynamic structures |
||||
- mixing int keys and string keys requires compatibility handling |
||||
- element access usually requires hash lookup or indirect access |
||||
- the memory layout is not contiguous, resulting in a lower CPU cache hit rate |
||||
- the value type is uncertain, possibly requiring dynamic type conversion before computation |
||||
- mechanisms such as copy-on-write and reference counting add extra runtime cost |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
$sum = 0; |
||||
foreach ($numbers as $n) { |
||||
$sum += $n; |
||||
} |
||||
``` |
||||
|
||||
If `$numbers` is an ordinary PHP Array, the compiler cannot confirm that every element is definitely an int. Even if it is known from the business logic to be `int[]`, the underlying code still needs to keep dynamic type handling capability. |
||||
|
||||
## std Strongly-Typed Containers |
||||
|
||||
Swoole AOT provides `std` containers to express "the structure and element type of this container are definite at compile time." |
||||
|
||||
Currently supported: |
||||
|
||||
- `std::array` |
||||
- `std::vector` |
||||
- `std::ordered_map` |
||||
- `std::map` |
||||
|
||||
Their goal is not to fully replace PHP Array, but to be used in performance-sensitive, structurally stable, and clearly-typed code paths. |
||||
|
||||
## std::array |
||||
|
||||
`std::array` is a fixed-length array whose length and element type are determined at compile time. |
||||
|
||||
```php |
||||
function main(): void |
||||
{ |
||||
$array = std::array(Type::Int, 100); |
||||
|
||||
$array[0] = 123; |
||||
$array[99] = 456; |
||||
|
||||
var_dump($array[0]); |
||||
} |
||||
``` |
||||
|
||||
Characteristics: |
||||
|
||||
- fixed length |
||||
- supports bounds checking |
||||
- fixed element type |
||||
- supports nested structures |
||||
- suited for matrices, fixed-length buffers, and fixed-structure data |
||||
|
||||
Nested example: |
||||
|
||||
```php |
||||
function main(): void |
||||
{ |
||||
$matrix = std::array( |
||||
std::array(Type::Int, 4), |
||||
3 |
||||
); |
||||
|
||||
$matrix[0][0] = 10; |
||||
$matrix[2][3] = 99; |
||||
|
||||
var_dump($matrix[2][3]); |
||||
} |
||||
``` |
||||
|
||||
`std::array` supports copy of the same type: |
||||
|
||||
```php |
||||
function main(): void |
||||
{ |
||||
$a = std::array(Type::Int, 3); |
||||
$b = std::array(std::array(Type::Int, 3), 2); |
||||
|
||||
$b[1][0] = 10; |
||||
$b[1][1] = 20; |
||||
$b[1][2] = 30; |
||||
|
||||
$a = $b[1]; // allowed, types are exactly identical, performs a std::array copy |
||||
} |
||||
``` |
||||
|
||||
## std::vector |
||||
|
||||
`std::vector` is a dynamically-sized contiguous array. |
||||
|
||||
```php |
||||
function main(): void |
||||
{ |
||||
$vector = std::vector(Type::Int); |
||||
|
||||
$vector[] = 1; |
||||
$vector[] = 2; |
||||
$vector[] = 3; |
||||
|
||||
var_dump($vector[1]); |
||||
var_dump(count($vector)); |
||||
} |
||||
``` |
||||
|
||||
An initial length can also be specified: |
||||
|
||||
```php |
||||
$vector = std::vector(Type::Float, 1024); |
||||
``` |
||||
|
||||
Characteristics: |
||||
|
||||
- dynamic length |
||||
- contiguous memory |
||||
- suited for a large number of elements of the same type |
||||
- better access performance than PHP Array |
||||
- fixed element type |
||||
|
||||
Vectors of the same type can be copied: |
||||
|
||||
```php |
||||
$a = std::vector(Type::Int); |
||||
$b = std::vector(Type::Int); |
||||
|
||||
$b[] = 10; |
||||
$b[] = 20; |
||||
|
||||
$a = $b; // allowed, types are exactly identical, performs a container copy |
||||
``` |
||||
|
||||
### Modifying Elements in foreach |
||||
|
||||
When iterating over `std::vector`, `std::map`, or `std::ordered_map`, you can update the values of existing elements, for example using `+=`: |
||||
|
||||
```php |
||||
foreach ($vector as $index => $value) { |
||||
$vector[$index] += 10; |
||||
} |
||||
``` |
||||
|
||||
During iteration you cannot perform structural modifications that may invalidate the C++ iterator, including appending elements, inserting or overwriting keys, `unset()`, and replacing the container as a whole. The compiler reports these directly as errors. When the structure needs to change, record the keys to be processed first and apply the modifications uniformly after the `foreach` ends. |
||||
|
||||
## std::ordered_map |
||||
|
||||
`std::ordered_map` is an ordered key-value container. |
||||
|
||||
```php |
||||
function main(): void |
||||
{ |
||||
$map = std::ordered_map( |
||||
Type::String, |
||||
Type::Int |
||||
); |
||||
|
||||
$map["a"] = 1; |
||||
$map["b"] = 2; |
||||
|
||||
var_dump($map["a"]); |
||||
} |
||||
``` |
||||
|
||||
Characteristics: |
||||
|
||||
- fixed key type |
||||
- fixed value type |
||||
- suited for scenarios requiring a stable key-value structure |
||||
- supports string keys and int keys |
||||
|
||||
Example: |
||||
|
||||
```php |
||||
$map = std::ordered_map(Type::Int, Type::Float); |
||||
|
||||
$map[10] = 1.25; |
||||
$map[20] = 3.5; |
||||
``` |
||||
|
||||
ordered_map of the same type can be copied: |
||||
|
||||
```php |
||||
$a = std::ordered_map(Type::Int, Type::Int); |
||||
$b = std::ordered_map(Type::Int, Type::Int); |
||||
|
||||
$b[10] = 100; |
||||
$a = $b; |
||||
``` |
||||
|
||||
## std::map |
||||
|
||||
`std::map` is a hash-table key-value container. |
||||
|
||||
```php |
||||
function main(): void |
||||
{ |
||||
$map = std::map( |
||||
Type::Int, |
||||
Type::Int |
||||
); |
||||
|
||||
$map[100] = 1; |
||||
$map[200] = 2; |
||||
|
||||
var_dump($map[100]); |
||||
} |
||||
``` |
||||
|
||||
Characteristics: |
||||
|
||||
- fixed key type |
||||
- fixed value type |
||||
- suited for a large number of key-value lookups |
||||
- usually used for mapping scenarios that do not require ordering |
||||
|
||||
map of the same type can be copied: |
||||
|
||||
```php |
||||
$a = std::map(Type::Int, Type::Int); |
||||
$b = std::map(Type::Int, Type::Int); |
||||
|
||||
$b[1] = 42; |
||||
$a = $b; |
||||
``` |
||||
|
||||
## Supported Element Types |
||||
|
||||
Type symbols: |
||||
|
||||
```php |
||||
Type::Int |
||||
Type::Float |
||||
Type::Bool |
||||
Type::String |
||||
Type::Array |
||||
Type::Object |
||||
Type::Any |
||||
Type::Stream |
||||
Type::Box |
||||
``` |
||||
|
||||
Class names can also be used as the value type: |
||||
|
||||
```php |
||||
class User |
||||
{ |
||||
} |
||||
|
||||
$vector = std::vector(User::class); |
||||
$array = std::array(User::class, 10); |
||||
$map = std::ordered_map(Type::String, User::class); |
||||
``` |
||||
|
||||
Class-typed containers check the object type at write time to prevent mixing in incorrect objects. |
||||
|
||||
## Conversion with PHP Array |
||||
|
||||
When a std container is assigned to an ordinary variable, it is automatically converted to a PHP Array: |
||||
|
||||
```php |
||||
function main(): void |
||||
{ |
||||
$vector = std::vector(Type::Int); |
||||
$vector[] = 1; |
||||
$vector[] = 2; |
||||
|
||||
$array = $vector; // converted to PHP Array |
||||
|
||||
var_dump(is_array($array)); // true |
||||
} |
||||
``` |
||||
|
||||
If the lvalue itself is a std container of the same type, a container copy is performed instead of converting to a PHP Array: |
||||
|
||||
```php |
||||
$a = std::vector(Type::Int); |
||||
$b = std::vector(Type::Int); |
||||
|
||||
$a = $b; // std::vector copy |
||||
``` |
||||
|
||||
If the types differ, the copy is not allowed: |
||||
|
||||
```php |
||||
$a = std::vector(Type::Int); |
||||
$b = std::vector(Type::Float); |
||||
|
||||
$a = $b; // compile failure |
||||
``` |
||||
|
||||
## UnsafePtr and native Function Parameters |
||||
|
||||
For scenarios where a std container reference needs to be passed between native functions, the `UnsafePtr` parameter can be used. |
||||
|
||||
The caller does not need to explicitly create an unsafe pointer: |
||||
|
||||
```php |
||||
function update(UnsafePtr $ptr): void |
||||
{ |
||||
$vector = std::unsafe_cast( |
||||
std::vector(Type::Int), |
||||
$ptr |
||||
); |
||||
|
||||
$vector[0] = 100; |
||||
} |
||||
|
||||
function main(): void |
||||
{ |
||||
$vector = std::vector(Type::Int, 1); |
||||
update($vector); // the compiler automatically converts to an UnsafePtr box |
||||
} |
||||
``` |
||||
|
||||
Rules: |
||||
|
||||
- `UnsafePtr` can only be used as a native function or method parameter |
||||
- the argument must be a std container variable |
||||
- local variables are never `UnsafePtr` |
||||
- the second argument of `std::unsafe_cast()` must be an `UnsafePtr` parameter in the current function signature |
||||
- the runtime validates the container type ID; inconsistent types throw an exception |
||||
|
||||
This ensures that unsafe casts do not depend on user-written temporary code and also avoids propagating unsafe pointers in local variables. |
||||
|
||||
## A Brief Introduction to the Compilation Principle |
||||
|
||||
An ordinary PHP Array is usually represented as a dynamic structure during AOT compilation: |
||||
|
||||
```cpp |
||||
php::Array |
||||
php::Var |
||||
``` |
||||
|
||||
This means every access needs to preserve PHP's dynamic semantics. |
||||
|
||||
std containers are different. The compiler records container metadata while parsing the code: |
||||
|
||||
- container type |
||||
- element type |
||||
- key type |
||||
- class type |
||||
- dimension information of `std::array` |
||||
- type ID |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
$vector = std::vector(Type::Int); |
||||
``` |
||||
|
||||
can generate something like: |
||||
|
||||
```cpp |
||||
php::StdVector<php::Int> vector; |
||||
``` |
||||
|
||||
For another example: |
||||
|
||||
```php |
||||
$array = std::array(std::array(Type::Int, 3), 2); |
||||
``` |
||||
|
||||
can generate something like: |
||||
|
||||
```cpp |
||||
php::StdArray<php::StdArray<php::Int, 3>, 2> array; |
||||
``` |
||||
|
||||
Therefore the compiler can directly generate strongly-typed access code: |
||||
|
||||
```php |
||||
$array[1][2] = 100; |
||||
``` |
||||
|
||||
which corresponds approximately to: |
||||
|
||||
```cpp |
||||
array[safeIndex(1, 2)][safeIndex(2, 3)] = 100; |
||||
``` |
||||
|
||||
This brings several benefits: |
||||
|
||||
- type conversion is determined at compile time |
||||
- shorter container access paths |
||||
- more compact element layout |
||||
- the C++ compiler can further optimize |
||||
- errors surface earlier at compile time |
||||
- friendlier to performance-sensitive code |
||||
|
||||
## Usage Recommendations |
||||
|
||||
Scenarios suitable for std containers: |
||||
|
||||
- large-scale numeric computation |
||||
- fixed-structure data |
||||
- a large number of elements of the same type |
||||
- array access in high-frequency loops |
||||
- mapping tables with stable key/value types |
||||
- hot paths that need to reduce the dynamic overhead of PHP Array |
||||
|
||||
Scenarios not suitable for std containers: |
||||
|
||||
- highly dynamic data structures |
||||
- frequently changing key/value types |
||||
- requiring full compatibility with PHP Array behavior |
||||
- flexible object structures at the business layer |
||||
- data from external input with an unstable structure |
||||
|
||||
The recommended approach is: keep using PHP Array or objects at the business boundary, and use std containers inside performance hot spots. |
||||
|
||||
## Performance Tests |
||||
### PHP Array |
||||
Test code: |
||||
```php |
||||
$u = (int)$argv[1]; |
||||
echo "u: $u\n"; |
||||
$r = rand(0, 10000); |
||||
$a = array_fill(0, 10000, 0); |
||||
|
||||
$begin = microtime(true); |
||||
for ($i = 0; $i < 10000; $i++) { |
||||
for ($j = 0; $j < 100000; $j++) { |
||||
$a[$i] += $j % $u; |
||||
} |
||||
$a[$i] += $r; |
||||
} |
||||
|
||||
echo $a[$r] . "\n"; |
||||
$end = microtime(true); |
||||
echo "sec: " . ($end - $begin) . "\n"; |
||||
``` |
||||
Test result: |
||||
```bash |
||||
php examples/array-loop/jit.php 999999 |
||||
u: 999999 |
||||
4999953010 |
||||
sec: 67.638107061386108 |
||||
``` |
||||
|
||||
### std::array |
||||
Test code: |
||||
```php |
||||
use native_types; |
||||
|
||||
function main(int $argc, array $argv): void |
||||
{ |
||||
$u = (int)$argv[2]; |
||||
echo "u: $u\n"; |
||||
$r = rand(0, 10000); |
||||
$a = std::array(Type::Int, 10000); |
||||
|
||||
$begin = microtime(true); |
||||
for ($i = 0; $i < 10000; $i++) { |
||||
for ($j = 0; $j < 100000; $j++) { |
||||
$a[$i] += $j % $u; |
||||
} |
||||
$a[$i] += $r; |
||||
} |
||||
|
||||
echo $a[$r] . "\n"; |
||||
$end = microtime(true); |
||||
echo "sec: " . ($end - $begin) . "\n"; |
||||
} |
||||
``` |
||||
|
||||
Test result: |
||||
```shell |
||||
./main examples/array-loop/main.php 999999 |
||||
u: 999999 |
||||
4999950397 |
||||
sec: 6.3918659687042236 |
||||
``` |
||||
|
||||
### C++ Test |
||||
Test code: |
||||
```cpp |
||||
#include <iostream> |
||||
#include <vector> |
||||
#include <cstdlib> |
||||
#include <ctime> |
||||
#include <chrono> |
||||
|
||||
int main(int argc, char* argv[]) { |
||||
std::srand(static_cast<unsigned>(std::time(nullptr))); |
||||
|
||||
long u = std::stoi(argv[1]); |
||||
std::cout << "u: " << u << "\n"; |
||||
|
||||
long r = std::rand() % 10001; |
||||
std::vector<long> a(10000, 0); |
||||
|
||||
auto begin = std::chrono::high_resolution_clock::now(); |
||||
|
||||
for (int i = 0; i < 10000; i++) { |
||||
for (int j = 0; j < 100000; j++) { |
||||
a[i] += j % u; |
||||
} |
||||
a[i] += r; |
||||
} |
||||
|
||||
std::cout << a[r] << "\n"; |
||||
|
||||
auto end = std::chrono::high_resolution_clock::now(); |
||||
std::chrono::duration<double> diff = end - begin; |
||||
std::cout << "sec: " << diff.count() << "\n"; |
||||
|
||||
return 0; |
||||
} |
||||
``` |
||||
Test result |
||||
```bash |
||||
g++ examples/array-loop/loop.cc -o loop -O3 |
||||
./loop 999999 |
||||
u: 999999 |
||||
4999954742 |
||||
sec: 6.22351 |
||||
swoole@swoole-26:~/workspace/aot/compiler$ |
||||
``` |
||||
|
||||
### Conclusion |
||||
The `std::array` container provided by the `AOT` compiler is almost `10` times as fast as `PHP Array`, and its performance is completely consistent with C++'s `std::vector`. |
||||
|
||||
## Summary |
||||
|
||||
PHP Array is a general, flexible, and highly expressive data structure, but its dynamism brings costs in type specification and performance. |
||||
|
||||
Swoole AOT's std containers provide a path better suited for compiler optimization: |
||||
|
||||
- use `std::array` to express fixed-length strongly-typed arrays |
||||
- use `std::vector` to express dynamic contiguous strongly-typed arrays |
||||
- use `std::ordered_map` / `std::map` to express strongly-typed mappings |
||||
- an ordinary variable receiving a std container is automatically converted to a PHP Array |
||||
- std containers of the same type support native copy |
||||
- UnsafePtr supports safely passing container references between native functions |
||||
|
||||
They let PHP code maintain high readability while providing the AOT compiler with sufficiently clear type information, thereby achieving more stable and more predictable performance. |
||||
@ -0,0 +1,48 @@ |
||||
# Test Coverage Checklist |
||||
|
||||
`bin/analyze-test-coverage.php` generates a coverage checklist from the source of PHPT and compiler PHPUnit fixtures. It is a static test-intent analysis tool and does not replace test execution. |
||||
|
||||
## Usage |
||||
|
||||
```bash |
||||
# terminal summary |
||||
php bin/analyze-test-coverage.php |
||||
|
||||
# reviewable full matrix |
||||
php bin/analyze-test-coverage.php \ |
||||
--format=markdown \ |
||||
--output=build/test-coverage.md |
||||
|
||||
# for CI or other tools to read |
||||
php bin/analyze-test-coverage.php \ |
||||
--format=json \ |
||||
--output=build/test-coverage.json \ |
||||
--strict |
||||
``` |
||||
|
||||
By default it scans `tests/compiler`, `phpunit/src`, and `phpunit/code`. One or more PHPT files or directories can also be passed at the end of the command; `--no-phpunit` analyzes only PHPT, and `--php-versions=8.4,8.5` sets the PHP version columns of the matrix. |
||||
|
||||
`--strict` returns a non-zero status when there are unexpected source-parse failures or unresolvable PHPUnit fixture references. Samples in negative data providers that are intentionally unparseable by php-parser are recorded separately under `expected_parser_diagnostics` and are not disguised as tool failures. |
||||
|
||||
## Three categories of coverage evidence |
||||
|
||||
Each applicable `PHP version × feature` row records: |
||||
|
||||
- `positive_compile`: valid PHPT, or positive PHPUnit compile fixtures; |
||||
- `runtime_semantics`: valid PHPT containing `EXPECT`, `EXPECTF`, or `EXPECTREGEX`; |
||||
- `negative_diagnostic`: PHPT expecting diagnostics, or PHPUnit tests/data providers that explicitly expect failure. |
||||
|
||||
`XFAIL` and unconditional `SKIPIF` are not counted on any evidence axis. PHP version ranges are inferred from test titles, `PHP_VERSION_ID` conditions in `SKIPIF`, and version strings in PHPUnit data rows. |
||||
|
||||
## Denominator |
||||
|
||||
The report only gives ratios with an explicit denominator: |
||||
|
||||
- AST node coverage denominator: the concrete AST node kinds provided by the currently installed `nikic/php-parser`; `Expr_Error`, used for error recovery, is not counted. |
||||
- Feature-axis coverage denominator: the number of rows in the feature catalog with `introduced <= target PHP version`. Each of the positive-compile, runtime-semantics, and negative-diagnostic axes is computed independently. |
||||
|
||||
The tool does not combine the three axes of different meanings into a single "overall project coverage". The full JSON preserves the feature catalog, per-item evidence sources, matrix, AST node occurrence counts, parse issues, and exclusion reasons, for further inspection by CI. |
||||
|
||||
## Classification boundaries |
||||
|
||||
AST nodes are extracted automatically by the parser. Semantic features that cannot be distinguished by nodes alone (such as DNF occurrence positions, property hook variants, `exit(message: ...)`) are supplemented by the explicit feature catalog in the analyzer. When adding a new language feature, register its introduction version and detection rule at the same time to keep the version matrix's denominator explicit. |
||||
@ -0,0 +1,462 @@ |
||||
# TypePHP WASM Technical Plan and Implementation Roadmap |
||||
|
||||
> Status: WASI 0.2 Component and Chrome Worker prototypes implemented |
||||
> Research date: 2026-08-07 |
||||
> Current goal: WASI 0.2 (Preview 2), NTS, single-threaded; WASI 0.1 not supported |
||||
|
||||
## 1. Document Purpose |
||||
|
||||
This document records the technical decisions, functional boundaries, runtime architecture, primary risks, validation methods, and phased implementation plan for TypePHP's WebAssembly support. |
||||
|
||||
The implementation validation completed on 2026-08-07 has proven that a trimmed PHP 8.5, the PHPX core, TypePHP-generated code, GMP, MPFR, and mpdecimal can be statically linked into a single module via the WASI SDK and run in Wasmtime. See [Building a TypePHP WASI Program](WASI_BUILD.md) for the reproducible build procedure. The remainder of this document also retains the browser-stage design goals. |
||||
|
||||
## 2. Core Conclusions |
||||
|
||||
The first TypePHP WASM release adopts the following path: |
||||
|
||||
```text |
||||
PHP source code |
||||
-> TypePHP compiler |
||||
-> TypePHP-generated C++ |
||||
-> WASI SDK compilation and static linking |
||||
+ PHP NTS |
||||
+ PHPX |
||||
+ TypePHP runtime |
||||
+ GMP / MPFR / mpdecimal |
||||
+ PHP embed/WASI runtime |
||||
-> typephp.wasm (WASI 0.2 command component) |
||||
``` |
||||
|
||||
Specific decisions are as follows: |
||||
|
||||
1. The first release reuses the current C++/Zend backend; it does not directly generate WAT/WASM, nor does it reimplement the PHP runtime. |
||||
2. Use the WASI SDK `wasm32-wasip2` sysroot to generate the Component directly; Chrome uses Jco to transpile it to ESM, without maintaining a second Emscripten ABI. |
||||
3. PHP, PHPX, TypePHP-generated code, and the high-precision libraries are all statically linked into a single `.wasm` module. |
||||
4. Wasmtime and Chrome together provide CLI, stdio, exit, clocks, random, and a controlled filesystem; the Chrome host always runs inside a Worker. |
||||
5. Only PHP NTS is supported; threads are not supported. |
||||
6. Fiber and Generator are disabled. |
||||
7. C++ exceptions and the `setjmp/longjmp` required by Zend bailout must be supported. |
||||
8. Keep the PHP stream framework and local streams, and disable network transports and features that depend on OS process capabilities. |
||||
9. WordPress Playground and other PHP-WASM projects serve only as a source of patches and porting experience, not as a dependency or codebase for TypePHP. |
||||
|
||||
This document describes the shortest viable path. For the long-term backend-neutral approach, see [BACKEND_NEUTRAL_IR.md](BACKEND_NEUTRAL_IR.md). The WASI prototype proves that the TypePHP frontend and semantic layers do not need to be rewritten for WASM. |
||||
|
||||
## 3. Why Not Adopt WordPress Playground |
||||
|
||||
WordPress Playground is a mature browser-based WordPress product, but it is not a lightweight PHP-WASM porting layer. Its repository and build system simultaneously serve: |
||||
|
||||
- Multiple PHP versions and extension combinations; |
||||
- WordPress distributions and their assets; |
||||
- Browser, Web Worker, and Node.js runtimes; |
||||
- Virtual filesystems, mounting, and persistence; |
||||
- Network proxying and browser HTTP adaptation; |
||||
- NPM packages, a website, developer tooling, and integration tests; |
||||
- WordPress-specific APIs and product features. |
||||
|
||||
TypePHP cannot directly reuse the PHP-WASM binary published by Playground, because TypePHP needs to statically link PHPX, the compiled C++, and the high-precision libraries together. If TypePHP forked Playground, it would also be bound to Playground's monorepo, Node/NPM build, version matrix, and product release cycle. |
||||
|
||||
Therefore, the following principles are adopted: |
||||
|
||||
- Do not fork WordPress Playground; |
||||
- Do not make `@php-wasm/*` a runtime dependency of TypePHP; |
||||
- Do not copy its WordPress, network proxy, file sync, and UI layers; |
||||
- Only study the PHP configure parameters, php-src patches, Emscripten compatibility handling, and the minimal C API; |
||||
- All borrowed patches must be split apart, have their sources attributed, and be verified to still apply to TypePHP's pinned PHP/Emscripten versions. |
||||
|
||||
Projects such as `seanmorris/php-wasm` and `soyuka/php-wasm` follow the same principle: they may serve as build references and issue indexes, but they do not become TypePHP's base repository. |
||||
|
||||
## 4. Goals and Non-Goals |
||||
|
||||
### 4.1 Current WASI Goals |
||||
|
||||
- Load TypePHP compilation artifacts in WASI runtimes such as Wasmtime. |
||||
- Execute the statically compiled TypePHP application entry point. |
||||
- Preserve TypePHP's current primary language semantics based on Zend and PHPX. |
||||
- Correctly handle the PHP request lifecycle, C++ exceptions, and Zend bailout. |
||||
- Support the GMP, MPFR, and mpdecimal high-precision types. |
||||
- Support the WASI filesystem and the necessary local PHP streams. |
||||
- Return deterministic, testable errors for unsupported features, rather than link failures or runtime crashes. |
||||
- Make the build process reproducible, with pinned php-src, WASI SDK, and numeric library versions. |
||||
|
||||
### 4.2 First-Phase Non-Goals |
||||
|
||||
- Direct browser execution without host adaptation. |
||||
- pthread, Web Worker-parallel PHP, or shared memory. |
||||
- Fiber and Generator. |
||||
- Dynamic extension loading. |
||||
- Runtime compilation of PHP source code or general-purpose `eval()`. |
||||
- TCP, UDP, Unix sockets, and listening ports. |
||||
- Network clients such as MySQL, PostgreSQL, and Redis. |
||||
- Network protocol implementations such as `curl`, FTP, and SMTP. |
||||
- `fork`, `exec`, `system`, `shell_exec`, `proc_open`, and signal handling. |
||||
- FFI, JIT, opcache, and the debugger. |
||||
- Full WordPress compatibility. |
||||
- Implementing asynchronous host calls in the first phase. |
||||
|
||||
## 5. Target Platform Selection |
||||
|
||||
### 5.1 Currently Using the WASI SDK |
||||
|
||||
For now, establish a command-line-verifiable baseline first. The WASI SDK has already been verified to provide, simultaneously: |
||||
|
||||
- A complete C/C++ to WebAssembly toolchain; |
||||
- Standard Wasm C++ exception handling; |
||||
- The SJLJ required by Zend bailout; |
||||
- A capability-based filesystem; |
||||
- libc, time, and random number interfaces. |
||||
|
||||
PHP, PHPX, and all TypePHP C++ translation units must use consistent Wasm EH/SJLJ parameters. The linker must treat inconsistent function signatures as fatal errors. |
||||
|
||||
### 5.2 Chrome Component Host |
||||
|
||||
Chrome currently cannot natively instantiate a Component. The builder uses Jco to transpile the same WASI 0.2 Component into core Wasm and ESM, and `examples/wasm-hello/typephp-worker.mjs` demonstrates the host entry point. The browser adaptation does not include PHP, PHPX, or high-precision type semantics. |
||||
|
||||
## 6. Artifacts and Runtime Model |
||||
|
||||
### 6.1 Release Artifacts |
||||
|
||||
The recommended minimal release artifacts are: |
||||
|
||||
```text |
||||
dist/ |
||||
├── typephp.wasm |
||||
└── typephp-wasm.mjs |
||||
``` |
||||
|
||||
All C/C++ code goes into `typephp.wasm`. The browser does not automatically provide WASI imports on its own; `typephp-wasm.mjs` serves as the loading entry point for the WASI host/adapter, responsible only for: |
||||
|
||||
- Fetching and instantiating the `.wasm`; |
||||
- Providing stdout/stderr; |
||||
- Initializing the in-memory filesystem; |
||||
- Implementing or wiring in host capabilities such as WASI clocks and random numbers; |
||||
- Invoking the exported TypePHP lifecycle interfaces; |
||||
- Converting status codes and error messages into JavaScript results. |
||||
|
||||
PHP semantics, Zend object operations, or TypePHP business logic should not be placed in the JavaScript loader. |
||||
|
||||
### 6.2 Lifecycle |
||||
|
||||
The recommended model is "module startup once, request repeatable": |
||||
|
||||
```text |
||||
instantiate wasm |
||||
-> typephp_wasm_module_startup() |
||||
-> typephp_wasm_request_startup() |
||||
-> TypePHP AOT entry |
||||
-> typephp_wasm_request_shutdown() |
||||
-> request can be executed again |
||||
-> typephp_wasm_module_shutdown() |
||||
``` |
||||
|
||||
Each request must have an independent PHP request memory pool. Successful execution, PHP exceptions, C++ exceptions, and Zend bailout must all enter a unified cleanup path. |
||||
|
||||
The module-exported API can start from the following minimal set; the names are subject to the actual implementation: |
||||
|
||||
```c |
||||
int typephp_wasm_module_startup(void); |
||||
int typephp_wasm_run(int argc, const char **argv); |
||||
const char *typephp_wasm_last_error(void); |
||||
void typephp_wasm_module_shutdown(void); |
||||
``` |
||||
|
||||
`typephp_wasm_run()` executes the statically linked AOT entry point; it is not responsible for parsing and compiling arbitrary PHP source code at runtime. |
||||
|
||||
## 7. PHP Build Strategy |
||||
|
||||
### 7.1 Base Configuration |
||||
|
||||
- Pin a specific php-src commit, rather than pinning only a branch name. |
||||
- NTS build. |
||||
- Disable existing SAPIs such as CLI, CGI, FPM, and Apache. |
||||
- Add a minimal `typephp_wasm` SAPI, or first validate the lifecycle with a minimal embed prototype before converging on a dedicated SAPI. |
||||
- Disable opcache/JIT. |
||||
- Statically link all extensions. |
||||
- Disable unneeded extensions and auto-detection to prevent the host environment from changing build results. |
||||
- Use `config.site` and a separate patch directory to record cross-compilation conclusions. |
||||
|
||||
For the first phase, do not directly copy other projects' complete configure parameters. Start from a minimal PHP core, and add extensions one by one according to TypePHP PHPT and runtime dependencies. |
||||
|
||||
### 7.2 Extension Layering |
||||
|
||||
It is recommended to divide extensions into three groups: |
||||
|
||||
1. **Must enable**: core, standard, SPL, date, pcre, hash, json, etc., required for TypePHP and Zend basic operation; the final set is subject to actual linking and test results. |
||||
2. **Optional local extensions**: ctype, filter, mbstring, tokenizer, fileinfo, zlib, etc., with no OS network dependencies, but they increase size. |
||||
3. **Disable in first phase**: sockets, curl, mysqli, PDO network drivers, pcntl, posix, FFI, shm, sysv, readline, opcache/JIT, etc. |
||||
|
||||
GMP, MPFR, and mpdecimal are first handled as static dependencies of the PHPX/TypePHP high-precision implementation; enabling PHP `ext/gmp` is not required. |
||||
|
||||
## 8. PHP Streams and OS Capabilities |
||||
|
||||
### 8.1 Do Not Disable the Entire Stream Subsystem |
||||
|
||||
The PHP standard library depends heavily on streams. Completely disabling streams would break file reads and writes, `php://`, include path handling, and some standard extensions, with little benefit and high compatibility cost. |
||||
|
||||
Keep in the first phase: |
||||
|
||||
- Ordinary file streams, backed by Emscripten MEMFS; |
||||
- `php://memory`; |
||||
- `php://temp`; |
||||
- Host mapping for `php://stdin`, `php://stdout`, and `php://stderr`; |
||||
- Whether `data://` is enabled is decided by size and security evaluation; |
||||
- Pure in-memory stream filters can be enabled as needed. |
||||
|
||||
### 8.2 Disable Network Streams |
||||
|
||||
The following should be disabled or not registered during the PHP build and runtime registration phases: |
||||
|
||||
- TCP, UDP, and Unix socket transports; |
||||
- The socket extension; |
||||
- Network-dependent wrappers such as `http://`, `https://`, and `ftp://`; |
||||
- `fsockopen()`, `pfsockopen()`, `stream_socket_*()`; |
||||
- Network database and network client extensions. |
||||
|
||||
In the first phase, PHP sockets should not be emulated via synchronous XHR or implicit JavaScript fetch. If HTTP is needed in the future, an explicit, authorizable asynchronous host API should be designed, rather than faking POSIX sockets. |
||||
|
||||
### 8.3 Other OS-Related Features |
||||
|
||||
The following capabilities must be disabled, degraded, or injected by the host: |
||||
|
||||
| Capability | First-phase strategy | |
||||
|---|---| |
||||
| Filesystem | MEMFS; optional read-only preloaded files | |
||||
| Current directory and paths | Virtual root directory; must not leak host paths | |
||||
| Environment variables | Loader-injected whitelist | |
||||
| Time | WASI clocks; the browser host implements this interface using browser clocks | |
||||
| Random numbers | WASI random; the browser host implements it using a secure random source, not a weak pseudo-random substitute | |
||||
| DNS, sockets | Not supported | |
||||
| Processes, shell | Not supported | |
||||
| Signals | Not supported | |
||||
| Users, groups, permissions | Fixed values or explicit errors | |
||||
| File locks | Cross-instance locks not supported in first phase; degrade as needed within a single instance | |
||||
| Persistence | Disabled by default; Chrome may explicitly enable OPFS filesystem snapshots | |
||||
|
||||
The compiler should progressively add WASM target capability checks: statically identifiable unsupported functions error at compile time; when a dynamic call cannot be statically determined, the runtime returns a deterministic error. These calls must never manifest as link-time missing symbols, empty functions, or undefined behavior. |
||||
|
||||
## 9. Exceptions, Bailout, and Cleanup |
||||
|
||||
This is the project's primary technical risk and must be validated before the full PHP feature port. |
||||
|
||||
### 9.1 Compilation Options |
||||
|
||||
When using native WebAssembly exceptions, C and C++ must use consistent `setjmp/longjmp` modes. The prototype is recommended to validate the following combination: |
||||
|
||||
```text |
||||
C compilation: |
||||
-sSUPPORT_LONGJMP=wasm |
||||
|
||||
C++ compilation: |
||||
-fwasm-exceptions |
||||
-sSUPPORT_LONGJMP=wasm |
||||
|
||||
Final link: |
||||
-fwasm-exceptions |
||||
-sSUPPORT_LONGJMP=wasm |
||||
``` |
||||
|
||||
All PHP, PHPX, TypePHP, and third-party C/C++ objects must use the same ABI and exception configuration. C++ exception catching cannot be enabled only at the final link stage. |
||||
|
||||
If target browser compatibility does not allow native Wasm EH, the Emscripten JavaScript exception mode can be researched as a fallback, but the two models must not be mixed in the same release. |
||||
|
||||
### 9.2 Boundary Rules |
||||
|
||||
- C++ exceptions must not cross exported functions unhandled into JavaScript. |
||||
- Zend bailout must be caught at the request top level and enter request shutdown. |
||||
- After bailout, dangling PHPX objects that depend on the destroyed request memory pool must not be destructed. |
||||
- PHPX `Variant`, `Object`, `Array`, and high-precision objects on the stack must complete destruction while the memory pool is still valid, or be taken over by a dedicated bailout-safe boundary. |
||||
- After one request fails, the next request must still be executable; otherwise the runtime can only be defined as a one-shot instance, which must be made explicit in the API. |
||||
|
||||
### 9.3 Must-Test Scenarios |
||||
|
||||
- PHP returns normally. |
||||
- PHP `throw` is caught by TypePHP code. |
||||
- An uncaught PHP exception reaches the request top level. |
||||
- `fatalError`/Zend bailout. |
||||
- C++ `throw` and `catch`. |
||||
- An exception is thrown when PHP calls C++ and C++ calls PHP again. |
||||
- PHPX objects and high-precision objects exist on the stack when bailout occurs. |
||||
- Execute success, failure, success three requests in sequence. |
||||
- Execute a request again after memory growth. |
||||
|
||||
## 10. Memory and High-Precision Libraries |
||||
|
||||
### 10.1 WASM Memory |
||||
|
||||
Use a single linear memory in the first phase, and validate `-sALLOW_MEMORY_GROWTH`. Record: |
||||
|
||||
- Initial memory; |
||||
- Maximum memory; |
||||
- PHP memory_limit; |
||||
- Zend memory reclamation after request end; |
||||
- Actual peak of the Emscripten allocator; |
||||
- Whether memory continues to grow after multiple requests. |
||||
|
||||
Do not choose `emmalloc` before benchmarking. PHP, GMP, MPFR, and mpdecimal are all allocation-intensive components; test size and runtime among candidates such as `dlmalloc` and `emmalloc`. |
||||
|
||||
### 10.2 GMP, MPFR, and mpdecimal |
||||
|
||||
- Statically compile all of them with the Emscripten toolchain. |
||||
- Disable assembly and host-CPU-specific optimizations. |
||||
- Pin limb, integer width, and ABI detection results. |
||||
- Do not depend on runtime dynamic library searching. |
||||
- Run existing BigInt, BigFloat, and Decimal PHPT, and add tests for maximum memory, division by zero, precision, rounding, and exception paths. |
||||
- Verify that library exceptions or allocation failures do not bypass PHP request cleanup. |
||||
|
||||
## 11. Recommended Repository Structure |
||||
|
||||
It is recommended to add a separate directory in the implementation phase, rather than scattering Emscripten conditionals into the existing build code: |
||||
|
||||
```text |
||||
wasm/ |
||||
├── README.md |
||||
├── build.sh |
||||
├── versions.env |
||||
├── config.site |
||||
├── cmake/ |
||||
│ └── TypePhpWasmToolchain.cmake |
||||
├── patches/ |
||||
│ ├── php-src/ |
||||
│ ├── gmp/ |
||||
│ ├── mpfr/ |
||||
│ └── mpdecimal/ |
||||
├── sapi/ |
||||
│ └── typephp_wasm/ |
||||
├── runtime/ |
||||
│ └── typephp-wasm.mjs |
||||
└── tests/ |
||||
``` |
||||
|
||||
Maintenance principles: |
||||
|
||||
- Patches should be small and independent, one patch per compatibility issue; |
||||
- Each patch records its upstream version, source, reason, and removal condition; |
||||
- Download caches are not committed to Git; |
||||
- php-src, Emscripten, and third-party libraries are locked by checksums; |
||||
- Build artifacts do not enter the source repository; |
||||
- CI keeps at least debug and release builds. |
||||
|
||||
## 12. Phased Implementation Plan |
||||
|
||||
### Phase 0: Toolchain Risk Validation |
||||
|
||||
Goal: Prove that the critical low-level mechanisms are feasible before integrating the full TypePHP. |
||||
|
||||
- Pin the Emscripten version. |
||||
- Compile a minimal mixed C/C++ program. |
||||
- Validate C++ exceptions. |
||||
- Validate `setjmp/longjmp`. |
||||
- Validate nesting and repeated invocation of both. |
||||
- Validate support across major browsers. |
||||
|
||||
Exit condition: exception and longjmp behavior is stable, with no unacceptable browser gaps. |
||||
|
||||
### Phase 1: Minimal PHP NTS |
||||
|
||||
Goal: PHP core completes the module and request lifecycle in the browser. |
||||
|
||||
- Cross-compile a minimal php-src. |
||||
- Implement a minimal WASM SAPI or embed validation layer. |
||||
- Support stdout/stderr and MEMFS. |
||||
- Execute a fixed entry point. |
||||
- Validate fatal errors, exceptions, and request shutdown. |
||||
|
||||
Exit condition: executing "success, failure, success" requests in sequence without crashes and without sustained memory growth. |
||||
|
||||
### Phase 2: Integrate PHPX and TypePHP |
||||
|
||||
Goal: The existing TypePHP C++ backend can be compiled by `em++` and statically linked. |
||||
|
||||
- Add a WASM platform/backend configuration to the compiler. |
||||
- Unify compilation flags across PHPX, TypePHP, and third-party libraries. |
||||
- Link a minimal TypePHP `main()`. |
||||
- Establish a WASM smoke PHPT subset. |
||||
- Add capability diagnostics for unsupported system APIs. |
||||
|
||||
Exit condition: basic type, function, class, exception, array, and object tests pass. |
||||
|
||||
### Phase 3: High Precision and Local Streams |
||||
|
||||
Goal: Support TypePHP's critical runtime capabilities. |
||||
|
||||
- Statically link GMP, MPFR, and mpdecimal. |
||||
- Run the full high-precision operator and boundary tests. |
||||
- Support the necessary `file://` and `php://` streams. |
||||
- Add a preloaded read-only resource mechanism. |
||||
- Clearly define all disabled wrappers, transports, and extensions. |
||||
|
||||
Exit condition: high-precision tests pass, local file behavior is deterministic, and all network APIs fail predictably. |
||||
|
||||
### Phase 4: Size, Performance, and Release |
||||
|
||||
Goal: Form a distributable TypePHP WASM SDK. |
||||
|
||||
- Release optimization and dead-code elimination. |
||||
- Review the exported symbol whitelist. |
||||
- Compare allocator and memory growth configurations. |
||||
- Establish download size, startup time, and peak memory benchmarks. |
||||
- Generate `typephp.wasm` and a thin `.mjs` loader. |
||||
- Write user-facing feature and limitation documentation. |
||||
|
||||
Exit condition: artifacts are reproducible, the compatibility checklist is complete, and performance reaches the preset baseline. |
||||
|
||||
### Phase 5: Optional Host Capabilities |
||||
|
||||
These are selected later based on real needs and are not default capabilities of the base runtime: |
||||
|
||||
- IDBFS or OPFS persistence; |
||||
- Explicit HTTP host API; |
||||
- Node.js host; |
||||
- WASI prototype; |
||||
- Multi-instance isolation; |
||||
- Web Worker parallel instances. |
||||
|
||||
Each capability must be enabled through an explicit capability, and PHP code must not obtain all host permissions by default. |
||||
|
||||
## 13. Testing Strategy |
||||
|
||||
### 13.1 Testing Layers |
||||
|
||||
1. **Toolchain tests**: exceptions, longjmp, static libraries, linking, and exported symbols. |
||||
2. **PHP lifecycle tests**: module/request startup, shutdown, bailout, and repeated requests. |
||||
3. **PHPX tests**: Variant, Object, Array, references, exceptions, and resource destruction. |
||||
4. **TypePHP PHPT**: select existing tests that do not depend on the OS, and maintain WASM skip reasons. |
||||
5. **High-precision tests**: full operators, boundaries, errors, and memory stress. |
||||
6. **Capability restriction tests**: network, processes, threads, and dynamic extensions must be stably rejected. |
||||
7. **Browser tests**: minimum supported versions for Chrome, Firefox, and Safari. |
||||
|
||||
### 13.2 Key Metrics |
||||
|
||||
- `.wasm` raw size and compressed size; |
||||
- First instantiation time; |
||||
- Module startup and request startup time; |
||||
- Execution time of a simple TypePHP program; |
||||
- Initial, peak, and post-multiple-request linear memory; |
||||
- Recoverability after exceptions and bailout; |
||||
- JavaScript loader size; |
||||
- Reproducible build checksums for identical inputs. |
||||
|
||||
## 14. Go/No-Go Conditions |
||||
|
||||
If any of the following occurs, pause the full port and re-evaluate the architecture: |
||||
|
||||
- A safe boundary between Zend bailout and C++ stack destruction cannot be established; |
||||
- Request failure stably corrupts subsequent requests, and the one-shot instance model is unacceptable; |
||||
- GMP, MPFR, or mpdecimal require a large-scale invasive fork; |
||||
- `.wasm` size or browser peak memory clearly exceeds the acceptable range of the target scenario; |
||||
- Safari, Firefox, and Chrome require mutually incompatible exception ABIs; |
||||
- Assumptions in PHPX that depend on native threads, dynamic linking, or OS resources cannot be isolated. |
||||
|
||||
If the shortest path is not feasible, then evaluate the standalone WASM runtime/backend described in [BACKEND_NEUTRAL_IR.md](BACKEND_NEUTRAL_IR.md); that rewrite should not be started prematurely without prototype data. |
||||
|
||||
## 15. External References |
||||
|
||||
- [PHP Source Repository](https://github.com/php/php-src) |
||||
- [Emscripten: C setjmp/longjmp Support](https://emscripten.org/docs/porting/setjmp-longjmp.html) |
||||
- [Emscripten: C/C++ Portability Notes](https://emscripten.org/docs/porting/guidelines/portability_guidelines.html) |
||||
- [Emscripten: Code and Memory Optimization](https://emscripten.org/docs/optimizing/Optimizing-Code.html) |
||||
- [WordPress Playground: Compiling PHP to WebAssembly](https://developer.wordpress.org/playground/developers/architecture/wasm-php-compiling/) |
||||
- [WordPress Playground Architecture](https://wordpress.github.io/wordpress-playground/developers/architecture/) |
||||
- [seanmorris/php-wasm](https://github.com/seanmorris/php-wasm) |
||||
- [soyuka/php-wasm](https://github.com/soyuka/php-wasm) |
||||
|
||||
These links are used to track upstream behavior and known porting issues. TypePHP's final implementation and compatibility must be verified by its own build, tests, and benchmarks, and must not directly inherit the conclusions of other projects. |
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,228 @@ |
||||
# Building TypePHP WASI Programs |
||||
|
||||
TypePHP uses the stable WASI 0.2 (Preview 2) and the Component Model. The C++ generated by TypePHP, the PHPX core, a trimmed PHP 8.5 NTS, GMP, MPFR, and mpdecimal are statically linked into a single `.wasm` command or library component. WASI 0.1 (Preview 1) is not supported. |
||||
|
||||
## Environment Requirements |
||||
|
||||
- WASI SDK 33 or higher (LLVM/Clang/LLD 22 or higher) |
||||
- PHP 8.4 or higher, used to run the TypePHP compiler |
||||
- Wasmtime 47 or higher, used to run and test artifacts |
||||
- Jco 1 or higher, used for the browser profile; the component profile does not need Jco |
||||
- wit-bindgen-cli 0.60.0, used for library/WasmExport mode; command mode does not need it |
||||
- The `wasm32-wasip2` integrated SDK bound to the current TypePHP version |
||||
|
||||
The WASI SDK's `bin` directory and Wasmtime must be added to the system `PATH`. The compiler does not probe or use conventional installation directories such as `/opt`, and does not accept dedicated tool directory configuration. WASI static libraries and headers are installed uniformly into PHPX's `wasm/wasm32-wasip2/`: |
||||
|
||||
```bash |
||||
export PATH="<wasi-sdk-bin>:<wasmtime-bin>:$PATH" |
||||
``` |
||||
|
||||
TypePHP uses the existing PHPX locating rules: first read `PHPX_HOME`, then the Composer `swoole/phpx` installation location, and finally `vendor/swoole/phpx`. No WASI-specific environment variable is added. |
||||
|
||||
The WASI build checks `wasm32-wasip2-clang`, `wasm32-wasip2-clang++`, `llvm-ar`, `llvm-ranlib`, `llvm-nm`, `wasm-component-ld`, and `wasmtime`, and confirms the target is `wasm32-unknown-wasip2`. The browser profile additionally checks `jco`, and library mode additionally checks a fixed version of `wit-bindgen`. All tools are found only from `PATH`; npm scripts automatically add the project-local `node_modules/.bin` to `PATH`. |
||||
|
||||
## One-command Build |
||||
|
||||
The source file for command mode must provide `main(): void`: |
||||
|
||||
```php |
||||
<?php |
||||
function main(): void |
||||
{ |
||||
echo "Hello from TypePHP/WASI\n"; |
||||
} |
||||
``` |
||||
|
||||
Run: |
||||
|
||||
```bash |
||||
php bin/tpc.php --wasm hello.php |
||||
``` |
||||
|
||||
Single-file input by default generates only a `hello.wasm` Component in the current directory that can be executed by Wasmtime, without requiring Jco. The generated `.cc` uses the same build directory rules as host mode, defaulting to the TypePHP root's `build/`; it can be overridden with `--build-dir <directory>`. |
||||
|
||||
A project can directly use `project.yml`: |
||||
|
||||
```yaml |
||||
name: wasm-hello |
||||
mode: bin |
||||
wasm: component |
||||
build-dir: build |
||||
output: component/wasm-hello.wasm |
||||
sources: |
||||
- src |
||||
``` |
||||
|
||||
`wasm` only accepts `component` or `browser`, not a boolean value. After configuring, directly run `php bin/tpc.php project.yml` to enter the WASI build without repeatedly passing `--wasm`. WASM projects default to `wasm32-wasip2` when `target-platform` is not configured; `build-dir`, `output`, and `wasm-browser-dir` are all resolved relative to the project file. See `examples/wasm-hello/` for a complete browser application, which explicitly uses `wasm: browser`. |
||||
|
||||
When you need to generate a browser module, configure `wasm: browser` and `wasm-browser-dir`, and ensure Jco is in `PATH`. |
||||
|
||||
The command line can also explicitly select the artifact: |
||||
|
||||
- `--wasm` or `--wasm=component`: generate only a Component runnable by Wasmtime, without checking Jco. |
||||
- `--wasm=browser`: generate the Component and the Jco browser module, requiring `jco` to be in `PATH`. |
||||
|
||||
Detailed configuration such as paths and sources continues to live in `project.yml`, not passed through `--wasm=`. |
||||
|
||||
PHP, PHPX, the TypePHP runtime, GMP, MPFR, and mpdecimal are precompiled into WASI static libraries during the SDK release stage. The application build only compiles the C++ generated by TypePHP for the current program, then links these `.a`. `tpc --wasm` does not download source code, nor does it invoke the build scripts of PHP, PHPX, or high-precision libraries. Library mode invokes `wit-bindgen-cli 0.60.0` from `PATH` to generate the current application's Canonical ABI bindings. |
||||
|
||||
PHP/WASI currently statically includes the `date`, `pcre`, `hash`, `json`, `lexbor`, `random`, `Reflection`, `SPL`, `standard`, `uri`, `ctype`, `calendar`, `bcmath`, `filter`, `tokenizer`, `mbstring`, `zlib`, `fileinfo`, `sodium`, `openssl`, `libxml`, `dom`, `SimpleXML`, `xml`, `xmlreader`, `xmlwriter`, `PDO`, `pdo_sqlite`, `zip`, `bz2`, and `exif` extensions. OpenSSL uses a crypto-only build without the TLS stream transport; HTTP/HTTPS is still provided by the WASI HTTP Component. |
||||
|
||||
Every C/C++ translation unit uniformly uses standard Wasm C++ exceptions and WASI SJLJ; the linking stage treats ABI warnings as errors, and old 32-bit `zend_long` caches are automatically invalidated. |
||||
|
||||
Run: |
||||
|
||||
```bash |
||||
wasmtime hello.wasm |
||||
``` |
||||
|
||||
Chrome Demo: |
||||
|
||||
```bash |
||||
cd examples/wasm-hello |
||||
npm ci |
||||
npm run wasm |
||||
npm run dev |
||||
``` |
||||
|
||||
The browser side always executes the Component in a dedicated Worker. The in-memory filesystem is used by default; after the startup message sent to the Worker sets `persistent: true`, the filesystem snapshot is restored and saved through OPFS at startup and exit. During program execution the synchronous in-memory filesystem is still used, avoiding crossing the asynchronous JS boundary on every PHP file access. |
||||
|
||||
## ZendVM Lifecycle for Command and Library |
||||
|
||||
### Command Mode |
||||
|
||||
Command mode has a generated C++ `main()` entry. The entry calls in sequence: |
||||
|
||||
```text |
||||
typephp_<project>_runtime_init(argc, argv) |
||||
→ php_embed_init() |
||||
→ PHP/SAPI module startup and MINIT |
||||
→ PHP request startup and RINIT |
||||
→ register and start the current TypePHP application module |
||||
→ MINIT and RINIT of the current application |
||||
|
||||
execute TypePHP main() |
||||
|
||||
typephp_<project>_runtime_shutdown() |
||||
→ RSHUTDOWN and module cleanup of the current application |
||||
→ php_embed_shutdown() |
||||
→ PHP request/module/SAPI shutdown |
||||
``` |
||||
|
||||
The caller does not need to be aware of these steps, because the generated native `main()` automatically wraps the entire program lifecycle. |
||||
|
||||
### Library Mode Must Create the Runtime Resource First |
||||
|
||||
A library component has no `main()` to execute automatically; merely instantiating the `.wasm` only completes the instantiation of the Component and the C/C++ Runtime, and does not mean the ZendVM request is already available. The Host must first call the generated WIT function: |
||||
|
||||
```wit |
||||
create-runtime: func() -> result<runtime, typephp-error>; |
||||
``` |
||||
|
||||
The corresponding call in the browser is: |
||||
|
||||
```js |
||||
const component = await instantiate(null, wasi.getImportObject()); |
||||
const runtime = await component.api.createRuntime(); |
||||
|
||||
try { |
||||
const result = await runtime.someExportedFunction(); |
||||
} finally { |
||||
runtime[Symbol.dispose](); |
||||
} |
||||
``` |
||||
|
||||
Internally, `createRuntime()` calls the project-level initialization symbol through `TYPEPHP_RUNTIME_INIT(<project>)(1, argv)`. The Host |
||||
only needs to call this one stable interface, and should not directly call `php_embed_init()`, MINIT, RINIT, or any Zend C API. |
||||
|
||||
The current initialization order is as follows: |
||||
|
||||
1. `php_embed_init()` initializes the Embed SAPI, the PHP core, and static extensions, and starts a PHP request; the PHP core and the already-registered static extensions complete MINIT/RINIT here. |
||||
2. Set up PHPX's exception bridge so that PHP exceptions can safely return into the generated WIT `result`. |
||||
3. Obtain the current TypePHP application's `zend_module_entry`, call `zend_register_module_ex()` and `zend_startup_module_ex()` to complete application module registration and MINIT. |
||||
4. Register standard streams and set SAPI request information such as the request path. |
||||
5. Because the Embed request and request memory pool are already started at this point, the generated code explicitly calls the current application module's `request_startup_func` to perform that module's RINIT; RINIT then initializes TypePHP request-level globals and class static data, and only returns the `runtime` resource after completion. |
||||
|
||||
What is "manually" called here is the Host-visible `create-runtime()`, not letting the user manually assemble the ZendVM lifecycle. The specific calls of MINIT/RINIT and their order are all encapsulated in PHPX and the generated Component adapter. |
||||
|
||||
### Exported Calls Share the Same Request |
||||
|
||||
All `#[WasmExport]` calls on the same `runtime` resource share the Zend request established by a single RINIT: |
||||
|
||||
- RINIT/RSHUTDOWN are not executed repeatedly before and after each function call. |
||||
- The PHP request memory pool, request-level globals, and static state persist until the resource is released. |
||||
- Currently only NTS is supported; calls on the same runtime must be serial, and the generated adapter rejects concurrent or re-entrant calls. |
||||
- Ordinary PHP exceptions are converted into WIT `result` errors, and the runtime can still continue to be used. |
||||
- A Zend bailout indicates that request state may already be corrupted, so the adapter marks the runtime as failed, and subsequent calls are rejected until the resource is released. |
||||
|
||||
### RSHUTDOWN Is Only Executed When the Resource Is Released |
||||
|
||||
Releasing the WIT `runtime` resource calls the project-level shutdown symbol through `TYPEPHP_RUNTIME_SHUTDOWN(<project>)()`: |
||||
|
||||
1. Call the current TypePHP application module's RSHUTDOWN to clean up TypePHP request-level objects and global data. |
||||
2. Unregister and shut down the current application module, performing the corresponding module cleanup. |
||||
3. Call `php_embed_shutdown()` to complete request shutdown, module shutdown, and SAPI shutdown of the remaining extensions. |
||||
4. Finally release the request memory pool to avoid PHP/CPP wrapper objects continuing to destruct after the memory pool disappears. |
||||
|
||||
Do not rely only on JavaScript GC to trigger the resource finalizer. Browser and Node Hosts should explicitly call `runtime[Symbol.dispose]()` in a `finally` block; Wasmtime or other Host bindings should also explicitly drop the resource. Directly terminating a Worker or process reclaims the entire Wasm instance, but does not guarantee that PHP's RSHUTDOWN/MSHUTDOWN callbacks are executed, so data that must be persisted should not be placed only in shutdown callbacks. |
||||
|
||||
A Component instance currently allows only one active runtime resource at a time. After release it can be recreated; when initialization fails or a Zend bailout occurs, the old resource should be released first, rather than continuing to call exported functions. |
||||
|
||||
## High-precision Types |
||||
|
||||
WASI artifacts include TypePHP's three language-level high-precision types: |
||||
|
||||
- `BigInt`: GMP 6.3.0 |
||||
- `BigFloat`: MPFR 4.2.2 |
||||
- `Decimal`: mpdecimal 4.0.1 |
||||
|
||||
A complete example is at [high-precision.php](../examples/high-precision.php). Build and run: |
||||
|
||||
```bash |
||||
php bin/tpc.php --wasm examples/high-precision.php |
||||
wasmtime -S http high-precision.wasm |
||||
``` |
||||
|
||||
Expected output: |
||||
|
||||
```text |
||||
1111111101111111110111111111010 |
||||
1000000000000000000000000000001 |
||||
12348.14159265358979324 |
||||
``` |
||||
|
||||
wasm32 uses 32-bit pointers, but PHP's `zend_long` remains 64-bit to maintain TypePHP's integer semantics with 64-bit PHP. GMP and mpdecimal use 32-bit limbs; this does not change arbitrary-precision semantics, but large-number throughput is lower than a native 64-bit build with assembly optimization. |
||||
|
||||
## Current Platform Boundaries |
||||
|
||||
- Only NTS, single-threaded is supported. |
||||
- Fiber and Generator are disabled; the compiler reports a fatal error directly when it finds `yield`. |
||||
- The PHPX Facade API is entirely disabled under `__wasi__`. PHPX core types and `phpx_std` are still usable. |
||||
- Dynamic extensions, network sockets, processes, shells, and signals are not supported. Statically recognizable calls report a fatal error at compile time. |
||||
- The PHP stream framework, local file capability, and time and random number capabilities provided by the WASI host are retained. |
||||
- A command component can be run directly by Wasmtime; a library component requires the Host to call `create-runtime()` and exported functions according to the WIT interface. Chrome uses the ESM generated by Jco and the Worker host in `examples/wasm-hello/typephp-worker.mjs`. |
||||
|
||||
The PHPX Facade is only a convenience wrapper generated for PHP optional extensions, not part of the TypePHP ABI. Disabling it entirely under WASI avoids exposing unavailable APIs such as curl, socket, and Swoole as interfaces that "compile but fail to link"; the PHP/WASI statically built-in extensions themselves are not affected by the Facade switch. |
||||
|
||||
## WASI SDK Directory |
||||
|
||||
The integrated SDK uses a single, complete prefix located at `wasm/wasm32-wasip2/` in the PHPX root directory: |
||||
|
||||
```text |
||||
phpx/wasm/wasm32-wasip2/ |
||||
├── include/php/ # PHP installed headers |
||||
├── include/phpx/ # PHPX and TypePHP runtime headers |
||||
├── include/gmp.h ... |
||||
├── lib/libphp.a |
||||
├── lib/libphpx.a |
||||
├── lib/libgmp.a |
||||
├── lib/libgmpxx.a |
||||
├── lib/libmpfr.a |
||||
├── lib/libmpdec.a |
||||
├── lib/libmpdec++.a |
||||
└── .typephp-wasi-sdk-abi |
||||
``` |
||||
|
||||
Ordinary users obtain this directory through the TypePHP/PHPX integrated installation package. TypePHP developers need to clone the `php-8.5.9-wasm` and PHPX source bound to the current version, and assemble the full SDK through `wasm/build-sdk.sh`. PHP/WASI is only responsible for PHP; PHPX is responsible for GMP, MPFR, its dedicated mpdecimal, and the PHPX runtime. All artifacts are installed into the same PHPX checkout. If PHPX is not at `vendor/swoole/phpx`, continue using the existing `PHPX_HOME` to point to that checkout. |
||||
|
||||
No path to separately override `libphp.a`, `libphpx.a`, or the numeric libraries is provided; all libraries, headers, and `.typephp-wasi-sdk-abi` must come from the same compatible build, avoiding mixing different `zend_long`, C++ exceptions, SJLJ, or Component Model ABIs. |
||||
@ -0,0 +1,143 @@ |
||||
# yield / Generator |
||||
|
||||
TypePHP compiles functions, methods, anonymous functions, and arrow functions that contain `yield` or `yield from` into Fiber generators. Calling them does not immediately execute the generator function body; instead it returns a global `\FiberGenerator` object. The Fiber is created and started only on the first iteration, or when `current()`, `valid()`, `send()`, or `throw()` is called. |
||||
|
||||
The generated C++ function body runs inside the Fiber. Each `yield` hands its key/value to the iteration driver via `Fiber::suspend()`, and resumes the original C++ call stack on `next()`, `send()`, or `throw()`. The runtime distinguishes not-started, running, suspended, normally-returned, and exceptionally-closed states via `NEW`, `RUNNING`, `SUSPENDED`, `CLOSED_RETURNED`, and `CLOSED_FAILED`. |
||||
|
||||
## Iteration interop |
||||
|
||||
The following combinations already work: |
||||
|
||||
| Driver | Iterated object | |
||||
|---|---| |
||||
| TypePHP Native `foreach` | TypePHP Native generator | |
||||
| TypePHP Native `foreach` | Zend `Generator` returned by dynamic PHP | |
||||
| ZendVM `foreach` | `\FiberGenerator` returned by a TypePHP Native generator | |
||||
| TypePHP `yield from` | arrays, `Iterator`, `IteratorAggregate`, Zend `Generator`, `\FiberGenerator` | |
||||
|
||||
`\FiberGenerator` implements `Iterator` with an interface signature consistent with PHP: |
||||
|
||||
```php |
||||
rewind(): void |
||||
next(): void |
||||
valid(): bool |
||||
current(): mixed |
||||
key(): mixed |
||||
send(mixed $value): mixed |
||||
throw(Throwable $exception): mixed |
||||
getReturn(): mixed |
||||
``` |
||||
|
||||
The current state machine already covers first and repeated `rewind()`, `next()`/`send()`/`throw()` when not started, calls after closure, automatic integer keys, normal and exceptional `getReturn()`, exception propagation, and executing `finally` when destructing in a suspended state. |
||||
|
||||
## Differences from PHP |
||||
|
||||
### Not a Zend Generator |
||||
|
||||
TypePHP generators return the global `\FiberGenerator`, not PHP's built-in `Generator`: |
||||
|
||||
```php |
||||
$generator instanceof Iterator; // true |
||||
$generator instanceof Generator; // false |
||||
``` |
||||
|
||||
Therefore the following differences exist: |
||||
|
||||
- A generator function cannot declare the precise return type `Generator`. |
||||
- `Iterator`, `Traversable`, `iterable`, `object`, `mixed`, or union types containing these compatible types can be used. |
||||
- `ReflectionGenerator` only accepts Zend `Generator` and cannot be used with `\FiberGenerator`. |
||||
- `get_class()`, Reflection class info, and the class name in exception stacks differ from Zend `Generator`. |
||||
- No guarantee that `var_dump()`, debug properties, or the internal object layout match Zend `Generator`. |
||||
- `\FiberGenerator` is a final runtime-internal type; business code is forbidden from directly instantiating, inheriting, cloning, or serializing it. |
||||
|
||||
### By-reference Generators not supported |
||||
|
||||
The following PHP syntax is not yet supported: |
||||
|
||||
```php |
||||
function &values(): iterable |
||||
{ |
||||
yield $value; |
||||
} |
||||
|
||||
foreach (values() as &$value) { |
||||
} |
||||
``` |
||||
|
||||
TypePHP does not support: |
||||
|
||||
- generator functions or methods returning by reference. |
||||
- by-reference yield semantics. |
||||
- by-reference `foreach` over a generator. |
||||
- maintaining element reference identity through a generator. |
||||
|
||||
`current()`, `send()`, `throw()`, and `getReturn()` all return ordinary PHP values; the runtime unwraps `INDIRECT` and `REFERENCE` wrappers and does not return reference containers. |
||||
|
||||
### Parameter limitations |
||||
|
||||
TypePHP generators do not yet support the following parameter declarations: |
||||
|
||||
- by-reference parameters, e.g. `function values(&$value)`. |
||||
- variadic parameters, e.g. `function values(...$values)`. |
||||
- by-reference variadic parameters, e.g. `function values(&...$values)`. |
||||
|
||||
Ordinary parameters, defaults, union-typed parameters, object parameters, and `$this` in methods can be used. Parameter type checks and constructor property promotion execute when the generator object is created, while the function body remains lazily executed. |
||||
|
||||
### Traversable boundary |
||||
|
||||
`yield from` and TypePHP Native object `foreach` use different underlying paths. `foreach` uniformly drives arrays, ordinary objects, and Zend `Traversable` through PHPX `ForeachIterator`; `yield from` still performs delegation within the generator itself. |
||||
|
||||
This means: |
||||
|
||||
- userland `Iterator`, `IteratorAggregate`, Zend `Generator`, and internal `Traversable` provided by extensions are all iterated through the class's `get_iterator` handler. |
||||
- ordinary objects directly traverse the live property table, performing public, protected, and private visibility checks under the current TypePHP class scope. |
||||
- `foreach ($iterable as $value)` that does not read the key does not call `Iterator::key()`. |
||||
- TypePHP `yield from` detects `IteratorAggregate::getIterator()` returning itself or forming an object cycle, and throws an exception. |
||||
- TypePHP Native `foreach` delegates `IteratorAggregate` unwrapping and cycle detection to the Zend iterator handler, keeping behavior consistent with the current PHP runtime. |
||||
- the exception type, message text, and stack info for an invalid `getIterator()` return value may not fully match ZendVM. |
||||
|
||||
### Fiber observable differences |
||||
|
||||
Zend `Generator` is a dedicated ZendVM execution object; TypePHP generators use PHP Fibers to save the full C/C++ stack, so: |
||||
|
||||
- the runtime environment must provide PHP Fibers. |
||||
- `Fiber`, internal closures, or TypePHP runtime frames may appear in exception stacks. |
||||
- file names, line numbers, and call-stack shapes are not guaranteed to exactly match `ReflectionGenerator` or Zend Generator. |
||||
- normal iteration, exception propagation, and `finally` during suspended destructors already have regression tests, but execution order for complex object cycles, request shutdown, process exit, and destructors that throw again may still differ from Zend Generator. |
||||
- when a Fiber is forcibly closed, Zend's internal graceful-exit is used to unwind the C++ stack; this object is not a public exception type that business code can catch or rely on. |
||||
|
||||
### yield from differences |
||||
|
||||
Key/value forwarding for arrays, ordinary Iterators, and generators, generator return values, and `send()`/`throw()` delegation are implemented, but the underlying implementation is not the Zend `yield from` opcode: |
||||
|
||||
- delegation is done through the `rewind()`, `valid()`, `key()`, `current()`, `next()`, `send()`, `throw()`, and `getReturn()` methods. |
||||
- side effects, exception stacks, and call counts produced by custom Iterator methods should avoid depending on Zend Generator's internal implementation details. |
||||
- `yield from` over a non-generator Iterator yields `null`; only Zend `Generator` and `\FiberGenerator` read `getReturn()`. |
||||
|
||||
## Performance differences |
||||
|
||||
Fiber generators do not change the generated code for ordinary array or ordinary container `foreach`. Extra cost arises only when a generator is actually created and driven. |
||||
|
||||
Each yield currently requires: |
||||
|
||||
- Fiber suspend/resume. |
||||
- `Iterator` method calls. |
||||
- generator state and object property reads/writes. |
||||
- creation and release of the key/value payload array. |
||||
- additional delegation calls in the `yield from` scenario. |
||||
|
||||
Therefore TypePHP Fiber generators are usually slower than Native C++ array `foreach`, and may also be slower than Zend's dedicated Generator opcode. High-frequency, short-element iteration should prefer arrays or Native containers; generators are more suited to lazy computation, streaming, and scenarios that need to preserve the full Native call stack. |
||||
|
||||
## Incompatibility checklist |
||||
|
||||
The following PHP behaviors currently cannot be relied upon: |
||||
|
||||
- the returned object being a Zend `Generator`. |
||||
- `instanceof Generator`. |
||||
- declaring the precise `Generator` return type. |
||||
- `ReflectionGenerator`. |
||||
- by-reference generator returns, by-reference yield, or by-reference foreach. |
||||
- by-reference or variadic generator parameters. |
||||
- debug output and internal properties identical to Zend Generator. |
||||
- all internal extension `Traversable` being iterable through Zend iterator handlers. |
||||
- Fiber closure, complex destruction, and process exit with exactly the same stack and destruction order as Zend Generator. |
||||
@ -0,0 +1,432 @@ |
||||
# AOT Compiler Optimization Priority Re-evaluation: Considering GCC/Clang Secondary Compilation |
||||
|
||||
## Core Principle |
||||
|
||||
The optimization value of AOT does not lie in doing what GCC/Clang already do, but in **providing information that GCC/Clang cannot infer from the C++ code**. |
||||
|
||||
What GCC/Clang can already do under `-O2`/`-O3`: |
||||
- Constant folding, constant propagation (SCCP) |
||||
- Dead code elimination (DCE) |
||||
- Common subexpression elimination (CSE) |
||||
- Loop unrolling, vectorization |
||||
- Instruction selection, register allocation |
||||
- Function inlining (within the same translation unit) |
||||
- Branch prediction optimization |
||||
- SIMD auto-vectorization |
||||
|
||||
What GCC/Clang **cannot do** — because the semantics are obscured by abstraction layers such as `php::Var`, `php::Object`, and virtual function calls: |
||||
- Narrow `php::Var` to a concrete C++ type |
||||
- Eliminate virtual calls of `php::Object` |
||||
- Eliminate reference counting operations |
||||
- Move `php::Array` from heap allocation to stack allocation |
||||
- Downgrade `php::BigInt` to native `int64_t` |
||||
- Eliminate the function lookup/dispatch overhead of `php::call` |
||||
- Cross-translation-unit global analysis (LTO can partially do this, but is limited by visibility) |
||||
|
||||
--- |
||||
|
||||
## Reordered Priorities |
||||
|
||||
### Tier 1: Directly Improve C++ Code Shape (GCC Cannot Fix) |
||||
|
||||
These optimizations change **what C++ code is generated**, not the C++ code that has already been optimized. This is the core value of AOT. |
||||
|
||||
#### #1 Type Inference → Generate Concrete C++ Types Instead of php::Var |
||||
|
||||
**Problem:** Currently AOT maps a large number of variables to `php::Var` (a general type), and GCC can only perform limited optimization on the member functions of `php::Var`. |
||||
|
||||
**Benefit:** |
||||
``` |
||||
// Currently generated code (GCC can't do much) |
||||
php::Var a = php::toInt(x); |
||||
php::Var b = php::toInt(y); |
||||
php::Var c = php::add(a, b); // GCC can't see this is addition |
||||
|
||||
// After type inference |
||||
int64_t a = php::toInt(x); |
||||
int64_t b = php::toInt(y); |
||||
int64_t c = a + b; // GCC can perform all integer optimizations |
||||
``` |
||||
|
||||
**Value amplification for GCC secondary compilation:** once the type becomes `int64_t`, GCC can further: |
||||
- Register allocation (no longer through `php::Var`'s memory layout) |
||||
- Constant folding and propagation |
||||
- Loop optimization (operations on `php::Var` inside loops become pure integer arithmetic) |
||||
- Automatic SIMD |
||||
|
||||
**Implementation recommendation:** this is the **highest priority**. A complete SSA form is not needed; just infer the most precise type at each assignment point and use that type to declare the variable during C++ code generation. The existing union/nullable type check infrastructure can be extended into type inference. |
||||
|
||||
--- |
||||
|
||||
#### #2 Devirtualization |
||||
|
||||
**Problem:** all non-final method calls in PHP are virtual calls. AOT generates `obj->method()` (a C++ virtual function call) even if only one subclass implements the method. |
||||
|
||||
**Benefit:** |
||||
``` |
||||
// Currently generated (virtual call, GCC dares not eliminate the vtable lookup) |
||||
return self->foo(); // virtual call |
||||
|
||||
// After devirtualization (GCC can inline this call!) |
||||
return Aot_MyClass_foo(self); // direct function call |
||||
``` |
||||
|
||||
**Breakthrough:** collect all classes and their inheritance relationships at compile time. If: |
||||
- The method is private → always call directly |
||||
- The method is final → always call directly |
||||
- The class has only 1 non-abstract implementation across all compiled files → call directly |
||||
- `self::foo()` calls a method of its own class → can call directly (if the class has no undiscovered subclasses) |
||||
|
||||
**Synergy with GCC:** once it becomes a direct call, GCC can: |
||||
- Inline the entire function body |
||||
- Cross-function constant propagation |
||||
- Chain-eliminate subsequent redundant operations |
||||
|
||||
--- |
||||
|
||||
#### #3 Escape Analysis → Stack Allocation + Reference Count Elimination |
||||
|
||||
**Problem:** every `new` object/array is allocated on the heap and its lifetime is managed through reference counting. |
||||
|
||||
**Benefit:** |
||||
``` |
||||
// Currently (heap allocation + refcount) |
||||
php::Array arr = php::newArray(); |
||||
arr.set("key", value); // refcount operations |
||||
return arr; // copy + refcount |
||||
|
||||
// After escape analysis (stack allocation + no refcount) |
||||
zend_array arr; // on the stack |
||||
zend_hash_update(&arr, "key", value); // no refcount |
||||
return php::Array::fromStack(std::move(arr)); // pack only at the return point |
||||
``` |
||||
|
||||
**Impact on GCC secondary compilation:** this is the optimization that helps GCC the **most**, because: |
||||
- Heap allocation → stack allocation: eliminates `malloc` calls, and GCC can fully optimize the stack layout |
||||
- Eliminating refcount: GCC does not need to analyze the side effects of interlocked operations and can freely reorder instructions |
||||
- GCC can perform SROA (Scalar Replacement of Aggregates) on stack variables, breaking arrays/objects into scalars |
||||
|
||||
**Implementation recommendation:** even simple local escape analysis (only analyzing whether an object is passed outside the function) can eliminate a large number of allocations. The complex version (cross-function escape analysis) continues to increase the benefit. |
||||
|
||||
--- |
||||
|
||||
#### #4 Call-Graph-Driven Cross-File Inlining |
||||
|
||||
**Problem:** the same PHP project may be compiled into multiple `.cc` files; GCC's LTO can inline across files but is limited by compilation time. |
||||
|
||||
**Benefit:** AOT knows the entire call graph at compile time and can **decide inlining while generating C++ code**: |
||||
|
||||
``` |
||||
// Currently |
||||
auto result = aot_smallHelper(x, y); // function call, even if the body is one line |
||||
|
||||
// After inlining |
||||
auto result = x + y; // GCC can continue optimizing |
||||
``` |
||||
|
||||
**Key decision information:** |
||||
- Function body size (< 10 lines → inline candidate) |
||||
- Call count (called only once → inlining can fully eliminate the function) |
||||
- Recursion marker (recursive functions are not inlined) |
||||
- Cross-function constant propagation opportunities (arguments are constants → after inlining GCC can fold the entire function) |
||||
|
||||
**Synergy with GCC:** AOT makes the "decision" (whether to inline) and generates the inlined code directly at the call site. GCC continues optimizing on the larger inlined body. AOT possesses information that GCC does not (a global view from the call graph). |
||||
|
||||
--- |
||||
|
||||
### Tier 2: Provide More Precise Type Information to GCC |
||||
|
||||
These optimizations improve the quality of the C++ types passed to GCC. |
||||
|
||||
#### #5 Integer Range Inference → Choose the Optimal Integer Type |
||||
|
||||
**Benefit:** |
||||
```php |
||||
// PHP source: loop counter, 0 to 100 |
||||
for ($i = 0; $i < 100; $i++) { ... } |
||||
|
||||
// Currently generated by AOT |
||||
int64_t i = 0; // always uses int64_t (to prevent overflow) |
||||
|
||||
// After range inference |
||||
int8_t i = 0; // 0..100 is enough, better cache locality |
||||
// or at least |
||||
int64_t i = 0; // but marked "no overflow possible", no BigInt promotion inserted |
||||
``` |
||||
|
||||
**Impact on GCC:** |
||||
- Smaller types → better vectorization (more elements packed into SIMD registers) |
||||
- "Will not overflow" assertions → GCC's VRP (Value Range Propagation) can perform more aggressive optimizations based on this premise |
||||
|
||||
#### #6 Avoid Unnecessary BigInt/BigFloat Allocation |
||||
|
||||
**Related to #5.** PHP's integer arithmetic automatically promotes to float or BigInt on overflow. If AOT can prove no overflow, no promotion code needs to be generated. |
||||
|
||||
``` |
||||
// Currently (every int operation must consider overflow) |
||||
php::Var result = php::BigInt::add(php::toBigInt(a), php::toBigInt(b)); |
||||
|
||||
// After range inference (no overflow) |
||||
int64_t result = a + b; // pure integer, GCC's world |
||||
``` |
||||
|
||||
--- |
||||
|
||||
### Tier 3: Partially Overlapping with GCC but Still Valuable |
||||
|
||||
#### #7 Constant Folding (PHP Level) |
||||
|
||||
**What GCC can do:** constant expressions at the C++ level are fully folded in GCC's SCCP pass. |
||||
|
||||
**AOT's unique value:** |
||||
- PHP-specific constants: `PHP_INT_MAX`, `PHP_VERSION`, `__DIR__`, etc. are fully resolved at compile time |
||||
- Constant resolution across PHP namespaces/class names (GCC cannot see PHP's symbol semantics) |
||||
- Results of PHP built-in functions: `strlen("hello")` → 5 (GCC does not know `strlen`'s internal implementation but can fold functions with known arguments) |
||||
|
||||
**Assessment:** limited value. Most PHP source code does not contain complex compile-time-evaluable constant expressions. |
||||
|
||||
#### #8 Dead Code Elimination (PHP Level) |
||||
|
||||
**What GCC can do:** GCC's DCE + unreachable block elimination is very mature. |
||||
|
||||
**AOT's unique value:** |
||||
- Eliminate branches based on the PHP type system: `if (false)` eliminated at the PHP level → no C++ code generated |
||||
- Eliminate branches based on type narrowing: `if ($x instanceof Foo)` when `$x`'s type is already determined to be `Bar` and `Foo ⊄ Bar`, the false branch is unreachable |
||||
- Eliminate uncalled PHP functions/classes (cross-file dead code) |
||||
|
||||
**Assessment:** medium value. AOT should focus on "DCE based on PHP semantics" rather than competing with GCC on "DCE based on C++ semantics". |
||||
|
||||
--- |
||||
|
||||
### Tier 4: Improve the Compilation Process Rather than Output Quality |
||||
|
||||
#### #9 File Caching (Compilation Acceleration) |
||||
|
||||
**What GCC can do:** ccache. But ccache requires file content hash matching. |
||||
|
||||
**AOT's unique value:** |
||||
- Cache PHP→C++ translation results (parsed AST + type information) |
||||
- Reuse across restarts |
||||
- Incremental updates (only recompile changed PHP files) |
||||
|
||||
**Assessment:** useful but not core. First optimize the output code quality, then optimize compilation speed. |
||||
|
||||
#### #10 Pass Pipeline Architecture |
||||
|
||||
Infrastructure at the engineering architecture level, improving code maintainability and extensibility. Does not directly affect output quality. |
||||
|
||||
--- |
||||
|
||||
## Priority Summary |
||||
|
||||
``` |
||||
Must do (directly affects the C++ types GCC sees): |
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ |
||||
⭐ #1 Type inference → avoid php::Var generalization 【largest single-point benefit】 |
||||
⭐ #2 Devirtualization → eliminate virtual call overhead 【second-largest benefit】 |
||||
⭐ #3 Escape analysis → stack allocation + eliminate refcount 【great benefit for numeric code】 |
||||
|
||||
Strongly recommended (give GCC better premises): |
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ |
||||
⭐ #4 Call-graph inlining → cross translation-unit boundaries |
||||
⭐ #5 Range inference → optimal integer type selection |
||||
⭐ #6 Eliminate unnecessary BigInt/BigFloat allocation |
||||
|
||||
Nice to have (unique value but not core): |
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ |
||||
#7 PHP-level constant folding |
||||
#8 PHP-semantic-level DCE |
||||
#9 Compilation cache acceleration |
||||
|
||||
Engineering foundation: |
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ |
||||
#10 Pass Pipeline architecture |
||||
``` |
||||
|
||||
## Further Analysis of #1 Type Inference |
||||
|
||||
This is the largest single-point benefit item and deserves further elaboration. The following is the concrete technical path: |
||||
|
||||
### Key Insight: No Complete SSA Needed |
||||
|
||||
AOT **generates C++ code**, not optimized bytecode. This means: |
||||
|
||||
1. No φ functions needed: C++ variable assignment is naturally the "fresh variable" form of SSA |
||||
2. No dominance frontier computation needed: the C++ compiler does it itself |
||||
3. Only need to compute the possible type of **each expression** and generate the corresponding C++ type declaration |
||||
|
||||
### Current Problem |
||||
|
||||
``` |
||||
PHP source: $a = 1; |
||||
$b = $a + 2; |
||||
Generated C++: php::Var a = php::toInt(php::toVar(1)); |
||||
php::Var b = php::add(a, php::toVar(2)); // !!all Var!! |
||||
``` |
||||
|
||||
What GCC sees at this point: |
||||
- All values are `php::Var` (a complex struct) |
||||
- Operations go through the `php::add()` function (external symbol, not inlinable) |
||||
- Cannot determine types, cannot narrow |
||||
|
||||
### Target |
||||
|
||||
``` |
||||
PHP source: $a = 1; |
||||
$b = $a + 2; |
||||
Generated C++: int64_t a = 1; |
||||
int64_t b = a + 2; // GCC can fully optimize |
||||
``` |
||||
|
||||
### Implementation Path |
||||
|
||||
**Phase A: Local type propagation (~300 lines)** |
||||
|
||||
Traverse the AST, at each assignment point: |
||||
1. If the RHS is a literal → type is clear, record it |
||||
2. If the RHS is a variable of known type → propagate the type |
||||
3. If the RHS is a binary operation and both operand types are known → infer the result type |
||||
4. If a variable is assigned different types multiple times → degrade to `php::Var` |
||||
|
||||
**Phase B: Conditional narrowing (~200 lines, depends on TypeSpecifier)** |
||||
|
||||
``` |
||||
if ($x instanceof MyClass) { // entering this branch, $x's value is determined to be MyClass |
||||
$x->method(); // direct call can be used here |
||||
} |
||||
``` |
||||
|
||||
**Phase C: Cross-basic-block merge (~200 lines)** |
||||
|
||||
``` |
||||
if (cond) { $x = 1; } else { $x = 2; } |
||||
// merge point: $x is int64_t (both branches are integers) |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Safety Constraints on Type Narrowing: PHP instanceof Semantics Analysis |
||||
|
||||
### Problem |
||||
|
||||
The user pointed out: the semantics of `if ($x instanceof Foo)` in PHP is that `$x` may be an instance of `Foo` **or any of its subclasses**. This is the same as C++'s `dynamic_cast<Foo*>`. |
||||
|
||||
This means: |
||||
|
||||
```php |
||||
if ($x instanceof Foo) { |
||||
$x->method(); // virtual call! $x's actual class may override method() |
||||
} |
||||
``` |
||||
|
||||
You **cannot** turn `$x->method()` into a direct call to `Foo::method()` just because it was narrowed to `Foo` — unless `Foo` is a final class or method is private. |
||||
|
||||
### Three Safety Levels of Narrowing |
||||
|
||||
#### Level 1: Type member accessibility (always safe) |
||||
|
||||
Even without knowing the exact type, as long as you know `$x` is a subtype of `Foo`, you can: |
||||
- Confirm `$x` has access to a certain method/property |
||||
- Eliminate impossible code paths (`if ($x instanceof Foo)` is false when `$x` is known to be `Bar` and `Foo` and `Bar` are unrelated) |
||||
- Generate correct C++ type annotations (`Foo*` instead of `void*`/`php::Var`) |
||||
|
||||
```php |
||||
// Before narrowing: $x is php::Var, calling foo() may throw an exception |
||||
// After narrowing: $x is some subtype of Foo, definitely has a foo() method |
||||
$x->foo(); // can confirm the method exists (though still a virtual call) |
||||
``` |
||||
|
||||
**Benefit:** eliminate runtime checks for nonexistent methods, improve error detection (found at compile time). |
||||
|
||||
#### Level 2: final class + private method (fully safe) |
||||
|
||||
| Scenario | Can narrow to exact type? | Can call directly? | |
||||
|------|-------------------|------------| |
||||
| `$x instanceof FinalClass` | Yes | Yes | |
||||
| `$x->privateMethod()` | Yes (private cannot be overridden) | Yes | |
||||
| `self::method()` and class has no subclasses | Yes | Yes | |
||||
| `$x instanceof NonFinalClass` | No | No | |
||||
|
||||
```php |
||||
final class Logger { |
||||
public function log(string $msg): void { ... } |
||||
} |
||||
|
||||
if ($x instanceof Logger) { |
||||
$x->log("test"); // safe: direct call to Aot_Logger_log($x, "test") |
||||
} |
||||
``` |
||||
|
||||
#### Level 3: Finite subclass set + guard (guard-based devirtualization) |
||||
|
||||
When `Foo` is not final, but AOT compiled the entire codebase and can determine **all subclasses of Foo**: |
||||
|
||||
``` |
||||
Known class hierarchy: |
||||
Foo (abstract) |
||||
├── SubFooA method() implementation |
||||
└── SubFooB method() implementation |
||||
``` |
||||
|
||||
At this point `$x instanceof Foo` means `$x` can only be `SubFooA` or `SubFooB`. It can generate: |
||||
|
||||
```cpp |
||||
// guard-based devirtualization |
||||
if (x.getInstanceOf(get_class(SubFooA))) { |
||||
Aot_SubFooA_method(x); // direct call |
||||
} else { |
||||
Aot_SubFooB_method(x); // direct call (the last one needs no check) |
||||
} |
||||
// replaces the original: x->method(); (virtual call) |
||||
``` |
||||
|
||||
**Benefit:** for class hierarchies with a known finite subclass set, replace vtable lookup with if-else dispatch. If there is only 1 subclass (de facto final), the branch is fully eliminated. |
||||
|
||||
### How PHP Itself Handles This |
||||
|
||||
The `zend_ssa_var_info` structure of Zend SSA precisely tracks this distinction: |
||||
|
||||
```c |
||||
typedef struct _zend_ssa_var_info { |
||||
uint32_t type; |
||||
zend_class_entry *ce; |
||||
bool is_instanceof : 1; // 0 = class == ce, 1 = may be child of ce |
||||
// ... |
||||
} zend_ssa_var_info; |
||||
``` |
||||
|
||||
- `is_instanceof = false`: the variable is **exactly this class** (from a `new Foo()` or a `get_class($x) === 'Foo'` check) |
||||
- `is_instanceof = true`: the variable **may be this class or its subclass** (from `$x instanceof Foo`) |
||||
|
||||
Only when `is_instanceof = false` and ce is non-abstract can it be safely devirtualized. |
||||
|
||||
### Impact on Tier 1 Optimizations |
||||
|
||||
| Optimization | Affected by instanceof semantics? | Valid scope | |
||||
|------|------------------------|---------| |
||||
| #1 Type inference (generate concrete C++ types) | Not directly affected | Narrowing to `Foo*` is still valuable (better than `php::Var`) | |
||||
| #2 Devirtualization | **Yes — instanceof does not provide exact type** | Only final classes, private methods, finite subclass guards | |
||||
| #3 Escape analysis | Not directly affected | Does not need exact type, only escape/non-escape determination | |
||||
| #4 Call-graph inlining | Not directly affected | Call-graph analysis concerns function relationships, unrelated to instanceof narrowing | |
||||
| #5 Range inference | Not directly affected | Integer ranges are unrelated to OOP inheritance | |
||||
|
||||
### Key Conclusion |
||||
|
||||
Although `instanceof` narrowing is not sufficient to deterministically devirtualize all calls, it provides the guarantee of **type member visibility**, which can eliminate method lookups, improve error detection, and generate more precise C++ type signatures (`Foo*` vs `php::Var`). |
||||
|
||||
The information that can truly drive devirtualization comes from: |
||||
1. **Final classes** — provided by source declarations |
||||
2. **Whole-program class hierarchy analysis** — after compiling all code, one knows which classes are de facto final (no subclasses in the codebase) |
||||
3. **Call-site-specific type inference** — tracking from `new Foo()` assignment to the call site |
||||
|
||||
--- |
||||
|
||||
## Conclusion |
||||
|
||||
Under the GCC/Clang secondary compilation architecture, the AOT compiler's optimization strategy should be: |
||||
|
||||
1. **"Freeze" PHP's dynamic type information into static C++ types** — this is what GCC cannot do itself |
||||
2. **"Land" virtual calls as direct calls** — so GCC can inline |
||||
3. **"Promote" heap allocation to stack allocation** — so GCC can perform SROA |
||||
4. **Do not compete with GCC on SSA-level optimization** — SCCP, DCE, CSE are left to GCC |
||||
5. **Focus on cross-translation-unit information** — call graph, class hierarchy — which even LTO has difficulty covering |
||||
@ -0,0 +1,569 @@ |
||||
# HHVM/HHBBC Compiler Review: Whole-Program Analysis, Type System, and Optimization Pipeline |
||||
|
||||
> 2026-06-11 · Based on source review of /home/swoole/workspace/cpp/hhvm |
||||
|
||||
--- |
||||
|
||||
## 1. Architecture Overview |
||||
|
||||
HHVM's compilation system is divided into three layers: |
||||
|
||||
| Layer | Component | Responsibility | |
||||
|------|------|------| |
||||
| Frontend | HackC (hphp/hack) | Hack/PHP source → HHAS (Hack Assembly) | |
||||
| Middle end | HHBBC (hphp/hhbbc) | HHBC bytecode → optimized HHBC bytecode | |
||||
| Backend | JIT (hphp/runtime/vm/jit) | HHBC → x86-64 machine code (runtime tracing JIT) | |
||||
|
||||
HHBBC (~80k lines of C++) is a **whole-program bytecode optimizer** based on fixed-point iterative analysis. Core files: |
||||
|
||||
| File | Lines | Purpose | |
||||
|------|------|------| |
||||
| `interp.cpp` | 6,685 | Abstract interpreter (forward dataflow) | |
||||
| `index.cpp` | 30,132 | Whole-program index: type info, dependency tracking, incremental re-analysis | |
||||
| `type-system.h` | 1,610 | Type lattice definition (trep + specialization) | |
||||
| `type-system.cpp` | 8,451 | Type operation implementation (meet/join/subtype/union) | |
||||
| `dce.cpp` | 3,102 | Type-aware dead code elimination (local + global) | |
||||
| `analyze.cpp` | 2,293 | Function-level dataflow analysis driver | |
||||
| `optimize.cpp` | 962 | Type-aware optimization pass | |
||||
| `cfg-opts.cpp` | ~300 | CFG optimization (unreachable block removal, exception edge simplification) | |
||||
|
||||
Comparison with AOT Compiler: |
||||
|
||||
| Dimension | HHBBC | AOT Compiler | |
||||
|------|-------|-------------| |
||||
| Compilation target | HHBC → optimized HHBC → JIT machine code | PHP → C++ → binary | |
||||
| Analysis level | Bytecode-level (whole-program) | AST-level (single-file) | |
||||
| IR form | php::Func (factored CFG + Bytecode) | PHP-Parser AST | |
||||
| Type inference | trep + specialization + fixed-point iteration | SSA + manual type annotations | |
||||
| Optimization scope | Whole-program (cross-file/cross-function) | Single function + class hierarchy | |
||||
| Call graph | Dynamically built (Index + assumption) | Static (classExtends + classMethodOverride) | |
||||
| Concurrency | Function-level parallel analysis | Serial execution | |
||||
| Runtime | HHVM (JIT + refcounting GC) | phpx (RAII wrapper around Zend API) | |
||||
| Code size | ~80k lines of C++ (HHBBC only) | ~15k lines of PHP | |
||||
|
||||
--- |
||||
|
||||
## 2. Reusable Innovative Designs |
||||
|
||||
### 2.1 Whole-Program Fixed-Point Analysis |
||||
|
||||
**Files**: `hphp/hhbbc/README`, `analyze.cpp`, `index.cpp` |
||||
|
||||
HHBBC's core algorithm: iterative convergence analysis across the **whole program**. |
||||
|
||||
``` |
||||
Algorithm flow: |
||||
1. Initialize work list = all functions/classes |
||||
2. Analyze each work unit in parallel (read-only Index, producing a new result) |
||||
3. Single-threaded: merge the new result into the Index |
||||
4. If the Index was updated → add back functions that depend on that info to the work list |
||||
5. Repeat until the Index reaches a fixed point (info no longer changes) |
||||
6. Final parallel optimization pass |
||||
``` |
||||
|
||||
Key design principles: |
||||
- Info in the Index can only **shrink** (shrinking types: becoming more precise) |
||||
- Types within function analysis can only **grow** (growing types: accumulated within the analysis context) |
||||
- Index info is **never wrong** (soundness guarantee) — it can only be insufficiently precise |
||||
|
||||
**AOT adoption priority: P0** |
||||
|
||||
Currently AOT is single-file serial compilation with no cross-file/cross-function analysis. What can be adopted: |
||||
|
||||
1. **Index structure**: store inferred results such as each function's return type, parameter types, and property types |
||||
2. **Dependency tracking**: record every query ("function F queried the return type of method M of class C") and add F back to the work list when C::M's type is updated |
||||
3. **Iterative convergence**: initialize all functions as "unknown" → analyze round by round → until type info stops changing |
||||
|
||||
Implementation suggestion: |
||||
```php |
||||
// Built in Preprocessor |
||||
class AnalysisIndex { |
||||
// Function return types (shrinking across iterations) |
||||
public array $returnTypes = []; // funcName => TypeInfo |
||||
|
||||
// Dependency graph: funcName => [depended-on funcName, ...] |
||||
public array $dependencies = []; |
||||
|
||||
// Function parameter states (collected from call sites) |
||||
public array $paramTypes = []; // funcName => [argIdx => TypeInfo] |
||||
|
||||
// Public static property states |
||||
public array $publicStaticProps = []; // className::prop => TypeInfo |
||||
} |
||||
``` |
||||
|
||||
Then repeatedly analyze functions in topological order until `$returnTypes` and `$paramTypes` stop changing. |
||||
|
||||
--- |
||||
|
||||
### 2.2 Trep Type Lattice (Bitset Type Lattice) |
||||
|
||||
**Files**: `hphp/hhbbc/type-system.h`, `type-system-bits.h`, `type-system-detail.h` |
||||
|
||||
HHBBC's type system is the cornerstone of the entire optimizer and is ingeniously designed: |
||||
|
||||
**Base mechanism — trep (type representation):** |
||||
```cpp |
||||
// Each "base type" is a bit |
||||
BUninit, BInitNull, BFalse, BTrue, BInt, BDbl, |
||||
BCls, BLazyCls, BFunc, BClsMeth, BEnumClassLabel, |
||||
BObj, BRes, BRFunc, BRClsMeth, |
||||
BSStr, BCStr, // uncounted/counted strings |
||||
BSVec, BCVec, BSVecE, BCVecE, BSVecN, BCVecN, // arrays' counted × empty dimensions |
||||
// ... Dict, Keyset follow the same pattern |
||||
``` |
||||
|
||||
Types are represented as union types via bitset combinations. For example, `Int|String` is `BInt|BStr`. |
||||
|
||||
**Specialization — extra info attached to types:** |
||||
``` |
||||
Int=n — known constant integer value |
||||
Dbl=n — known constant float value |
||||
{S,C}Str=s — known constant string value |
||||
Obj{<}=c — known class type (exact or subclass) |
||||
Arr(T1,T2,...) — known array shape (packed array) |
||||
Arr([T1:T2]) — known array key/value types |
||||
``` |
||||
|
||||
**Key innovations:** |
||||
- `counted` / `uncounted` dimension: strings and arrays distinguish whether they are reference-counted, allowing the compiler to optimize uncounted (static/constant) values more aggressively |
||||
- `empty` / `non-empty` dimension: arrays distinguish empty/non-empty, eliminating redundant empty checks |
||||
- Monotonic lattice: types can only change in one direction (Index types can only shrink, analysis types can only grow) |
||||
|
||||
**AOT adoption priority: P0** |
||||
|
||||
Currently AOT uses discrete constants like `TYPE_INT`, `TYPE_STRING`, `TYPE_ARRAY` with no fine-grained info such as union types or uncounted markers. It could: |
||||
|
||||
1. Use bitsets to represent union types (`int|string` instead of `mixed`) |
||||
2. Add an `Immutable` marker (values provably immutable at compile time) |
||||
3. Add a `NonEmpty` marker (arrays provably non-empty at compile time) |
||||
4. Build a type lattice defining meet (⊓) and join (⊔) operations |
||||
|
||||
```php |
||||
// Current |
||||
public const TYPE_INT = 1; |
||||
|
||||
// Suggested: bitset |
||||
class Type { |
||||
const BINT = 1 << 0; |
||||
const BSTRING = 1 << 1; |
||||
const BBOOL = 1 << 2; |
||||
const BFLOAT = 1 << 3; |
||||
// ... |
||||
const BIMMUTABLE = 1 << 16; // immutable marker |
||||
const BNONEMPTY = 1 << 17; // non-empty marker |
||||
|
||||
// union type: int|string = BINT | BSTRING |
||||
// meet: narrower = more specific |
||||
// join: wider = more general |
||||
} |
||||
``` |
||||
|
||||
--- |
||||
|
||||
### 2.3 Abstract Interpreter |
||||
|
||||
**Files**: `hphp/hhbbc/interp.cpp` (6,685 lines), `interp.h` |
||||
|
||||
HHBBC uses an **abstract interpreter** for function-level type inference. This differs from traditional SSA + constraint solving: |
||||
|
||||
``` |
||||
Algorithm (analyze_func): |
||||
1. Initialize the entry block's input state (parameter types = conservative assumptions from the Index) |
||||
2. Work list ← entry blocks |
||||
3. While work list not empty: |
||||
a. Pop a block |
||||
b. Run the abstract interpreter instruction-by-instruction over the block |
||||
c. On instructions that may throw → propagate current state to the exception edge |
||||
d. On branch instructions → propagate the post-branch state to the taken edge |
||||
e. Block may fallthrough → propagate final state to the fallthrough edge |
||||
f. If the target block's input state changed → add to work list |
||||
``` |
||||
|
||||
Key characteristics of the abstract interpreter: |
||||
- **State propagation**: compute a new `State` (locals + eval stack type info) after each instruction |
||||
- **Factored CFG**: exception edges are modeled separately (`FactoredExitBlock`), so ordinary instructions don't break basic blocks just because they "may throw" |
||||
- **Type specialization**: branch conditions automatically narrow types (e.g. `if (is_int($x))` → `$x` narrowed to `Int` in the then branch) |
||||
- **Constant propagation**: built-in constant folding and value tracking |
||||
|
||||
**AOT adoption priority: P1** |
||||
|
||||
Currently AOT's `SsaTypeOptimizer` does some type narrowing, but: |
||||
- No factored CFG concept (exceptions don't participate in dataflow) |
||||
- No abstract interpreter framework (each instruction defines `step(state) → new_state`) |
||||
- No iterative dataflow analysis |
||||
|
||||
A lightweight abstract interpretation can be implemented on top of SSA: |
||||
```php |
||||
class AbstractInterpreter { |
||||
// Run abstract interpretation over each SSA basic block |
||||
function analyzeBlock(Block $block, State $inputState): State { |
||||
foreach ($block->instructions as $instr) { |
||||
$inputState = $this->step($instr, $inputState); |
||||
} |
||||
return $inputState; |
||||
} |
||||
|
||||
// Each instruction defines how to transform state |
||||
function step(Instruction $instr, State $state): State { |
||||
// switch on instruction type |
||||
// return new State with updated types |
||||
} |
||||
} |
||||
``` |
||||
|
||||
--- |
||||
|
||||
### 2.4 Type-Aware Dead Code Elimination |
||||
|
||||
**File**: `hphp/hhbbc/dce.cpp` (3,102 lines) |
||||
|
||||
HHBBC's DCE differs from traditional liveness-based DCE — it **combines type analysis** to discover more dead code: |
||||
|
||||
``` |
||||
Two kinds of DCE: |
||||
1. Local DCE — within a single basic block |
||||
- Traverse the block backward |
||||
- Maintain a "reverse stack": mark which eval stack slots will be used in the future |
||||
- Unused stack slots → the instruction producing that slot can be deleted |
||||
- Unused local stores → eliminated |
||||
|
||||
2. Global DCE — across basic blocks |
||||
- Liveness analysis over locals |
||||
- Allows eliminating dead stores across blocks |
||||
``` |
||||
|
||||
Key design: **type awareness** — DCE needs to know each instruction's type to judge correctly. For example: |
||||
- If `$x` has type `Bottom` (unreachable) on some path, code using `$x` may be dead code |
||||
- If `$x` is a counted type, related inc/dec ref operations cannot be eliminated |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
Currently AOT's DCE essentially relies on GCC/Clang's `-O2`. Compile-time local DCE can be added: |
||||
- Eliminate assignments to unused local variables |
||||
- Eliminate side-effect-free pure computations (if the result is unused) |
||||
- Use type info to determine whether an operation may have side effects |
||||
|
||||
--- |
||||
|
||||
### 2.5 DataType Encoding: 3-of-7 Error-Correcting Code |
||||
|
||||
**File**: `hphp/runtime/base/datatype.h` |
||||
|
||||
HHVM's runtime type tags use an extremely clever bit encoding: |
||||
|
||||
```cpp |
||||
// DataType is uint8_t |
||||
// - bit 0 (LSB): countedness — 0 = definitely uncounted |
||||
// - bits 1-7: 3-of-7 error-correcting code — exactly 3 bits set to 1 |
||||
|
||||
// Type detection becomes simple bit operations: |
||||
// Check if Vec or Dict: dt <= KindOfVec |
||||
// Check if it has a persistent version: dt <= KindOfString |
||||
// Check if null/uninit: dt >= KindOfUninit |
||||
``` |
||||
|
||||
Characteristics of the 3-of-7 encoding: |
||||
- There are exactly C(7,3) = 35 8-bit values with exactly 3 bits set to 1 (each persistent/counted pair shares the same 3-of-7 code) |
||||
- Detecting any type: `(dt & type_mask) == type_tag` completes in two instructions |
||||
- Unsigned LT/GT comparisons implement efficient type-group detection |
||||
|
||||
**AOT adoption priority: P4** |
||||
|
||||
This informs the **quality of generated code** for the AOT compiler — but mainly at the phpx layer. Consider giving phpx's `Variant` type tags a more efficient encoding to optimize runtime type checks like `is_int()`/`is_string()`. |
||||
|
||||
--- |
||||
|
||||
### 2.6 RepoAuthType: Space-Efficient Type Storage in Bytecode |
||||
|
||||
**Files**: `hphp/runtime/base/repo-auth-type.h`, `repo-auth-type-tags.h` |
||||
|
||||
HHBBC encodes the type info derived from analysis as `RepoAuthType`, embeds it in the bytecode stream (`AssertRAT` instruction), and makes it available to the JIT. |
||||
|
||||
Design points: |
||||
- Compact encoding (`CompactTaggedPtr`): type tag + optional pointer (class name/array shape) packed into one pointer width |
||||
- Covers the complete lattice from `Uninit` (most precise) to `Cell` (most general) |
||||
- `SubObj` / `SubCls` tags support subclass relationships |
||||
- Array shape specialization (precise type of packed arrays) |
||||
|
||||
**AOT adoption priority: P3** |
||||
|
||||
Currently AOT's type annotations are represented through the C++ type system (`int64_t`, `php::string`, `php::array`). For typed properties, RAT-like thinking can be used to generate more precise C++ type declarations. |
||||
|
||||
--- |
||||
|
||||
### 2.7 Index Dependency Tracking |
||||
|
||||
**File**: `hphp/hhbbc/index.cpp` (30,132 lines) |
||||
|
||||
The Index is not just storage for type info — more centrally it provides the **dependency tracking mechanism**: |
||||
|
||||
``` |
||||
Dependency kinds (DependencyKind): |
||||
- ReturnTy — function return type |
||||
- ConstVal — constant value |
||||
- ClsConst — class constant |
||||
- PropType — property type |
||||
- PublicSProp — public static property type (especially important!) |
||||
``` |
||||
|
||||
When a function's return type is updated in the Index, all functions that queried that return type are marked for re-analysis. |
||||
|
||||
**Special handling of public static properties**: public static properties can be modified by any function, so the Index tracks all mutation operations. When analysis finds that a static property is never modified, more aggressive constant propagation becomes possible. |
||||
|
||||
**AOT adoption priority: P1** |
||||
|
||||
Complements the whole-program analysis in 2.1 — Index dependency tracking is the foundation for implementing iterative analysis. In AOT: |
||||
|
||||
```php |
||||
class Index { |
||||
// Return types: funcName => Type |
||||
// Dependencies: funcName => [depends_on_funcName, ...] |
||||
// Dirty flags: funcName => bool (needs re-analysis) |
||||
} |
||||
``` |
||||
|
||||
After Preprocessor scans all files, build the initial Index, then iterate until convergence. |
||||
|
||||
--- |
||||
|
||||
### 2.8 Factored CFG (Exception-Factored Control Flow Graph) |
||||
|
||||
**Files**: `hphp/hhbbc/cfg.h`, `parse.cpp` |
||||
|
||||
HHBBC's control flow graph **factors out** exception edges: it doesn't terminate basic blocks at every instruction that may throw, instead letting basic blocks be as large as possible. |
||||
|
||||
``` |
||||
Traditional CFG: |
||||
instr1 ; block 1 |
||||
instr2(may_throw) ; block 1 terminates here (because it may throw) |
||||
--- |
||||
instr3 ; block 2 |
||||
|
||||
Factored CFG: |
||||
instr1 ; block 1 |
||||
instr2(may_throw) |
||||
instr3 |
||||
; block 1 contains multiple instructions |
||||
; exception edges connect from the factored exit edge to the exception handler |
||||
``` |
||||
|
||||
Benefits: |
||||
- Larger basic blocks → more efficient dataflow analysis (fewer block boundaries) |
||||
- Type info can rule out exception possibilities → exception edges can be deleted during optimization |
||||
- Easier instruction scheduling during the JIT stage |
||||
|
||||
**AOT adoption priority: P4** |
||||
|
||||
For an AOT compiler (translating to C++ rather than directly generating machine code), the CFG is mostly handled by GCC. But for function coloring and SSA analysis, more precise exception modeling can be a reference. |
||||
|
||||
--- |
||||
|
||||
### 2.9 Parallel Analysis |
||||
|
||||
**Files**: `hphp/hhbbc/parallel.cpp` (69 lines), `README` |
||||
|
||||
In HHBBC's fixed-point iteration, the analysis of each work unit can be **fully parallel**: |
||||
|
||||
``` |
||||
Thread-safety model: |
||||
- Analysis phase: can only read the Index (internally thread-safe) + read php metadata (immutable) |
||||
- Merge phase: single-threaded Index update (no lock needed) |
||||
``` |
||||
|
||||
This leverages the "Index info is never wrong" property — even if two threads analyze based on different versions of the Index, merging results never produces wrong info. |
||||
|
||||
**AOT adoption priority: P3** |
||||
|
||||
Similar to KPHP's pipeline parallelism. In AOT, independent functions/classes can be analyzed in parallel. Note that Index merging must be serial (or use lock-free structures). |
||||
|
||||
--- |
||||
|
||||
### 2.10 Public Static Property Optimization |
||||
|
||||
**File**: `hphp/hhbbc/index.cpp` |
||||
|
||||
HHBBC tracks the mutation of every public static property across the whole program: |
||||
|
||||
``` |
||||
- Initial state: conservative assumption (may be modified by any function) |
||||
- During each analysis round, record which functions modify which static props |
||||
- If a static prop is never modified during analysis → can be constant-folded |
||||
- If a static prop is only ever assigned one type → its type can be narrowed |
||||
``` |
||||
|
||||
This is more precise than a simple "is it written" analysis because it is a **whole-program** analysis that can see cross-file mutations. |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
Currently in AOT, public static properties always use `Variant` (mixed type). Through whole-program analysis, static properties that are only assigned internally can be optimized to precise C++ types. |
||||
|
||||
--- |
||||
|
||||
## 3. Toolchain Analysis |
||||
|
||||
### 3.1 Test Infrastructure |
||||
|
||||
HHVM has a massive test system: |
||||
|
||||
| Layer | Directory | Count | Purpose | |
||||
|------|------|------|------| |
||||
| Quick test | `hphp/test/quick/` | 865 .php files | Fast regression tests | |
||||
| Slow test | `hphp/test/slow/` | 7,927 .php files | Comprehensive functional/performance tests | |
||||
| Zend test | `hphp/test/zend/` | ~4,500 | PHP compatibility tests (from php-src) | |
||||
| Ext test | `hphp/test/ext/` | ~800 | Extension feature tests | |
||||
| Server test | `hphp/test/server/` | ~100 | HTTP/RPC integration tests | |
||||
| HHBBC unit test | `hphp/hhbbc/test/` | 3 C++ files | Compiler internal tests | |
||||
| **Total** | | **~14,675** | | |
||||
|
||||
Test runner characteristics: |
||||
- Quick vs Slow layering: Quick (~1 second to run) for pre-commit, Slow (~10 minutes) for CI |
||||
- Supports multiple run modes: interp / JIT / hhbbc + JIT / RepoAuthoritative |
||||
- Zend tests: directly reuse php-src's official tests to verify PHP compatibility |
||||
|
||||
**AOT adoption:** |
||||
- Quick/Slow layered test strategy — classify the existing `tests/compiler/` by run time |
||||
- Directly reuse php-src's official PHPT tests — verify the AOT compiler's PHP behavior compatibility |
||||
- HHBBC has too few internal unit tests (only 3) — should not be emulated; AOT's PHPUnit coverage is better |
||||
|
||||
### 3.2 Hack Type Checker |
||||
|
||||
HHVM's Hack language has a complete **static type checker** (`hphp/hack/`) that runs independently of the compiler: |
||||
|
||||
- Compile-time type annotations (`int`, `string`, `vec<T>`, `dict<TK,TV>`, `shape(...)`) |
||||
- Gradual typing: can migrate gradually from unannotated code |
||||
- IDE integration (LSP protocol support) |
||||
- Type coverage tracking |
||||
|
||||
**AOT adoption:** |
||||
- Currently AOT uses annotations like `@phpstan-type`; phpstan can be integrated for pre-compilation type checking |
||||
- Type coverage is a useful metric: X% of functions/variables have precise type annotations |
||||
|
||||
### 3.3 Tracing / Debug Infrastructure |
||||
|
||||
HHVM has rich debugging and tracing: |
||||
- `TRACE_SET_MOD(hhbbc)` — module-level conditional logging (compile-time switch) |
||||
- `hphp/tools/` — various analysis tools (bytecode viewer, profiler, etc.) |
||||
- `debug.cpp` — human-readable printing of types/states |
||||
|
||||
**AOT adoption:** |
||||
- The current `-vv` verbose output can adopt the TRACE_MODULE approach to filter logs by module |
||||
- Visualization of type analysis results can aid debugging the optimizer |
||||
|
||||
### 3.4 RepoAuthoritative Mode |
||||
|
||||
HHVM supports a **compile once, deploy many times** mode: |
||||
|
||||
``` |
||||
Source → HHBBC whole-program analysis → optimized bytecode Repo → multiple processes load the Repo directly |
||||
``` |
||||
|
||||
- The Repo stores **pre-analyzed type info** (RepoAuthType) |
||||
- No re-inference of types at runtime |
||||
- Bytecode-level interning (string/class/function ids) |
||||
- Multiple processes share the same Repo (mmap) |
||||
|
||||
**AOT adoption: P4** |
||||
|
||||
Currently AOT directly generates `.cc` files and compiles them into a binary, already on the "compile once, deploy many times" path. But the Repo's mmap sharing idea can be used for constant pool sharing in multi-process environments (rather than each process loading independently). |
||||
|
||||
--- |
||||
|
||||
## 4. Type System Compatibility Analysis |
||||
|
||||
### 4.1 Hack vs PHP vs AOT Compiler |
||||
|
||||
| Feature | PHP 8.2 | Hack | AOT Compiler | |
||||
|------|---------|------|-------------| |
||||
| Base types | mixed, int, string, float, bool, array, null, void, never | int, string, float, bool, null, void, noreturn, mixed, dynamic, nonnull, nothing | Same as PHP | |
||||
| Union types | `int\|string` | `int\|string` (but discouraged in practice) | Supported | |
||||
| Intersection types | `X&Y` (8.1+) | Via `where` constraints | Not supported | |
||||
| Generics | None | `vec<T>`, `dict<TK,TV>`, `class Box<T>` | `std::vector<T>` (native C++) | |
||||
| Array types | `array` | `vec<T>`, `dict<TK,TV>`, `keyset<T>` | `php::array` | |
||||
| Shapes | None | `shape('x' => int, 'y' => string)` | None (object suggested) | |
||||
| Enums | enum (8.1) | enum, enum class (with label) | Supports PHP 8.1 enum | |
||||
| Nullable types | `?int` | `?int` (function params only) | Supported | |
||||
| nothing / bottom | None | Yes (empty function return, unreachable) | None | |
||||
| dynamic | None | Yes (selectively skip type checking) | None | |
||||
|
||||
### 4.2 Key Differences in the Type Lattice |
||||
|
||||
Hack/HHBBC's type system is far richer than PHP's: |
||||
|
||||
``` |
||||
HHBBC type lattice: |
||||
|
||||
Cell (any value) |
||||
| |
||||
InitCell (not Uninit) |
||||
| | |
||||
Prim Boxed types (Obj, Res, ...) |
||||
| |
||||
InitPrim (not null) |
||||
| | | |
||||
Num Bool Str/ArrKey |
||||
| | |
||||
Int Dbl |
||||
|
||||
Bottom: Bottom (no value - unreachable) |
||||
Top: Cell (any value) |
||||
``` |
||||
|
||||
Characteristics: |
||||
- **Bottom**: represents the type of unreachable paths, enabling dead code elimination |
||||
- **Counted/Uncounted**: static strings vs runtime-allocated strings, different lifetimes |
||||
- **Array shapes**: `dict<'name' => string, 'age' => int>` is a distinct type |
||||
- **Wait handle**: `WaitH<T>` represents the type of async results |
||||
|
||||
### 4.3 Hack's Gradual Typing Design |
||||
|
||||
Hack evolved from PHP and supports gradual migration: |
||||
- `mixed` — any type (equivalent to unannotated PHP) |
||||
- `dynamic` — any type, and no type errors reported (a looser mixed) |
||||
- `<<__Soft>>` — soft type hints (not enforced at runtime) |
||||
|
||||
This design allows large codebases to add type annotations incrementally, avoiding an "all or nothing" approach. |
||||
|
||||
### 4.4 Syntax Differences from the AOT Compiler |
||||
|
||||
| Feature | HHBBC Input (HHAS) | AOT Compiler | |
||||
|------|------|-------------| |
||||
| Base PHP version | Hack (PHP 5.6 branch) | PHP 8.2+ | |
||||
| Type annotations | Mandatory (required by Hack) | Optional (phpstan annotations) | |
||||
| Generics | `vec<T>`, `dict<K,V>`, custom generic classes | No native generics | |
||||
| Lambda / closures | `$x ==> $x + 1` (short lambda) | PHP closures (`function($x) { return $x + 1; }`) | |
||||
| async / await | Native support (WaitHandle) | None | |
||||
| Shapes | `shape('x' => int)` | None | |
||||
| Enum class | Supported (enum class with label) | PHP 8.1 enum only | |
||||
| XHP | HTML template syntax | None | |
||||
| Case types | `case type T = int \| string` | None | |
||||
|
||||
--- |
||||
|
||||
## 5. Summary and Priority Recommendations |
||||
|
||||
| Priority | Technique | Difficulty | Benefit | Notes | |
||||
|--------|------|------|------|------| |
||||
| **P0** | Whole-program fixed-point analysis | Very high | Very high | Cross-file type inference, return type narrowing, eliminating pseudo-dynamic calls | |
||||
| **P0** | Trep type lattice | Medium | Very high | Precise union types, uncounted markers, non-empty markers — improves precision of all optimizations | |
||||
| **P1** | Abstract interpreter | High | High | Replaces/enhances current SSA analysis, supports branch type narrowing and constant propagation | |
||||
| **P1** | Index dependency tracking | High | High | Foundation for whole-program analysis, incremental compilation | |
||||
| **P2** | Type-aware DCE | Medium | Medium | Eliminate more dead code, reduce generated C++ code size | |
||||
| **P2** | Public static prop optimization | Medium | Medium | Precise typing of global static properties | |
||||
| **P3** | RepoAuthType storage | Low | Medium | Embed precise type info in artifacts (limited value for the C++ generation stage) | |
||||
| **P3** | Parallel analysis | High | Medium | Faster compilation of large projects | |
||||
| **P4** | DataType encoding optimization | Low | Low | Micro-optimization of phpx Variant type checks | |
||||
| **P4** | Factored CFG | High | Low | C++ compiler already handles CFG optimization | |
||||
| **P5** | Repo mmap sharing | High | Low | Optimization for specific deployment scenarios | |
||||
|
||||
### Key Takeaways |
||||
|
||||
1. **HHBBC's whole-program analysis is its biggest differentiator** — this is the AOT compiler's greatest architectural gap. Single-file analysis cannot see cross-file type info, forcing many calls to use dynamic dispatch. |
||||
|
||||
2. **The type system is the core engine of optimization** — HHBBC's trep + specialization + monotonic lattice is a huge investment (about 12k lines), but it is the quality foundation for all optimization passes. |
||||
|
||||
3. **Dependency tracking is the key to incremental analysis** — the Index automatically records "who queried what", so only affected functions need re-analysis, avoiding re-analyzing the whole program each time. |
||||
|
||||
4. **Factored CFG and DataType encoding are more JIT-oriented** — these designs serve runtime JIT optimization. The AOT compiler generates C++ source, and GCC/Clang handle these low-level optimizations, so the investment should not be duplicated. |
||||
|
||||
5. **The test infrastructure is worth learning from** — Quick/Slow layering, directly reusing php-src tests, and multiple run modes all have direct reference value for building the AOT compiler's test CI. |
||||
@ -0,0 +1,451 @@ |
||||
# KPHP Compiler Review: Design, Optimization, Toolchain, and Syntax Compatibility Analysis |
||||
|
||||
> 2026-06-11 · Based on source review of /home/swoole/workspace/cpp/kphp |
||||
|
||||
--- |
||||
|
||||
## 1. Architecture Overview |
||||
|
||||
| Dimension | KPHP | AOT Compiler | |
||||
|------|------|-------------| |
||||
| Compilation target | PHP → C++ → binary | PHP → C++ → binary | |
||||
| IR form | Custom vertex (op_*) tree | PHP-Parser AST → C++ string | |
||||
| Type inference | Iterative convergence graph inference (converging toward generalization) | SSA + manual type annotations | |
||||
| Intermediate optimization | Multi-pass AST rewriting (~60+ pipes) | Direct AST → C++ translation + a few optimizations | |
||||
| Runtime | Self-developed (allocator/string/array/mixed) | phpx (C++ RAII wrapper around Zend API) | |
||||
| Concurrency model | Single thread + reactor/epoll | Depends on Zend/TSRM | |
||||
| Thread safety | None (lock-free memory allocator) | Zend TSRM | |
||||
| Code size (compiler) | ~18k lines of pipe code | ~6k lines of CompilerBase | |
||||
|
||||
--- |
||||
|
||||
## 2. Reusable Innovative Designs |
||||
|
||||
### 2.1 Rewrite Rules DSL (Pattern-Matching Optimization Rules) |
||||
|
||||
**File**: `compiler/rewrite-rules/early_opt.rules` |
||||
|
||||
A declarative DSL describes AST rewriting and generates C++ optimization code at compile time. The rule format is `(pattern) => (replacement)`, supporting conditional clauses and embedded C++ expressions: |
||||
|
||||
```lisp |
||||
;; strlen constant folding |
||||
(op_func_call {"strlen"} arg:(op_string)) |
||||
=> (op_int_const { std::to_string(arg->str_val.size()) }) |
||||
|
||||
;; explode index direct access → specialized version |
||||
(op_index (op_func_call {"explode"} delim s) k:(op_int_const)) |
||||
=> (op_func_call {"_explode_nth"} delim s k) |
||||
|
||||
;; ("" . $x) → (string)$x — eliminate meaningless concatenation |
||||
(op_concat (op_string {""}) x) => (op_conv_string x) |
||||
|
||||
;; substring type optimization: conv(substr(...)) → conv(_tmp_substr(...)) |
||||
(op_conv_int x) if let x2 { to_tmp_string_expr(x) } => (op_conv_int x2) |
||||
``` |
||||
|
||||
**AOT adoption priority: P0** |
||||
|
||||
Currently `FuncCallOptimizer`'s `strlen`/`count` optimizations are hardcoded. Introducing a similar rule engine can: |
||||
- Add optimizations declaratively, reducing maintenance cost |
||||
- Use `if let` conditions for local pattern variable binding |
||||
- Separate rule files from the compiler, making hot reloading feasible |
||||
|
||||
Implementation suggestion: implement a `RewriteRule` class at the PHP level and load/match it in `FuncCallOptimizer`. |
||||
|
||||
--- |
||||
|
||||
### 2.2 Smart instanceof / Smart Casts (Type Narrowing) |
||||
|
||||
**File**: `compiler/pipes/transform-to-smart-instanceof.cpp` |
||||
|
||||
After `if ($x instanceof A)`, `$x` is automatically renamed to `instance_cast<A>($x)` inside the if body. The core innovation is doing variable splitting **before type inference**: |
||||
|
||||
```php |
||||
// PHP source |
||||
if ($x instanceof A) { |
||||
$x->methodOfA(); // $x automatically becomes instance_cast<A>($x) |
||||
} |
||||
|
||||
// Reverse guard pattern |
||||
if (!($x instanceof A)) return; |
||||
// After this, $x is replaced by instance_cast<A>($x) across the whole function scope |
||||
``` |
||||
|
||||
It also handles the renaming of same-named variables across different `catch (SomeClass $e)` blocks to prevent assumption confusion. |
||||
|
||||
**AOT adoption priority: P0** |
||||
|
||||
Currently `SsaTypeOptimizer` only narrows int/float/string base types. Object type narrowing can be added: |
||||
1. Identify `instanceof` guards in the SSA builder |
||||
2. Replace with the target subclass type in the then/else branches |
||||
3. Combine with the existing `stableObjects` mechanism for devirtualization |
||||
|
||||
--- |
||||
|
||||
### 2.3 Pipeline Parallel Compilation |
||||
|
||||
**File**: `compiler/compiler.cpp` |
||||
|
||||
The compilation process works at function granularity, chaining pipes via `operator>>` and processing them in parallel across multiple threads: |
||||
|
||||
```cpp |
||||
SchedulerConstructor{scheduler} |
||||
>> PipeC<LoadFileF>{} |
||||
>> PipeC<FileToTokensF>{} |
||||
>> PipeC<ParseF>{} |
||||
>> PassC<GenTreePostprocessPass>{} |
||||
/* ... 60+ pipes */; |
||||
``` |
||||
|
||||
Three pipe types: |
||||
- **PipeC\<T\>**: general transformation, input→output |
||||
- **PassC\<T\>**: function-level transformation, traverses all AST vertices |
||||
- **SyncC\<T\>**: synchronization point, outputs only after all inputs are processed |
||||
|
||||
Different functions can be processed simultaneously at different stages; global storage uses thread-safe or lock-free structures. |
||||
|
||||
**AOT adoption priority: P3** |
||||
|
||||
Currently `Preprocessor` → `CompilerBase` is serial. For large projects, function-level parallelism can be introduced: |
||||
- Compile classes/functions independently at their granularity |
||||
- Use `SyncC` synchronization points to merge global symbol tables |
||||
|
||||
--- |
||||
|
||||
### 2.4 Switch Splitting (State Machine Transformation) |
||||
|
||||
**File**: `compiler/pipes/split-switch.cpp` |
||||
|
||||
Extract each case branch of a switch into an independent function, driven by a state variable: |
||||
|
||||
```cpp |
||||
// Each case becomes: |
||||
int case_state = 0; |
||||
auto case_res = switch_func_N(&case_state); |
||||
if (case_state == 1) return case_res; // normal return |
||||
if (case_state == -1) break; // break semantics |
||||
``` |
||||
|
||||
`break N` and `continue N` are converted to setting the state variable to `-1` + return, consistent with the `_brk_flag` / `_cnt_flag` approach previously implemented in AOT. |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
Large switches can be split into independent functions, reducing single-function complexity and giving GCC more room for inlining/optimization. |
||||
|
||||
--- |
||||
|
||||
### 2.5 Constant Immutability Markers and init-once |
||||
|
||||
**Files**: `compiler/pipes/collect-const-vars.cpp`, `runtime-common/core/memory-resource/` |
||||
|
||||
Compile-time constant arrays/strings use special refcount markers: |
||||
|
||||
```cpp |
||||
ExtraRefCnt::for_global_const // immutable, triggers COW on modification |
||||
ExtraRefCnt::for_instance_cache // shared across requests, not modifiable |
||||
``` |
||||
|
||||
These constants are stored in the data section, initialized once at server startup, and used read-only in subsequent requests. Any modification automatically triggers COW. |
||||
|
||||
**AOT adoption priority: P1** |
||||
|
||||
Currently AOT already promotes constant arrays to static variables, but a finer-grained immutability marker mechanism can be introduced to reduce unnecessary COW copies (when the compiler can prove a variable is never modified). |
||||
|
||||
--- |
||||
|
||||
### 2.6 Function Specialization (Multi-Version Generation) |
||||
|
||||
**File**: `compiler/pipes/early-optimization.cpp` |
||||
|
||||
Functions are specialized by their arguments before type inference: |
||||
|
||||
- `microtime()` → `_microtime_float()` or `_microtime_string()` (based on the true/false argument) |
||||
- `list() + explode()` → `_explode_tupleN()` (precise N-tuple type) |
||||
- `explode()[N]` → `_explode_nth()` (O(1) direct access to the Nth element) |
||||
- `substr()` in a function argument position → `_tmp_substr()` (avoid string copying) |
||||
|
||||
The key is that **specialized versions return more precise types**. For example, `microtime()` returns `mixed`, while `_microtime_float()` returns `float`. |
||||
|
||||
**AOT adoption priority: P1** |
||||
|
||||
`FuncCallOptimizer` currently only does constant folding; it can be extended to **multi-version specialization**: |
||||
|
||||
```php |
||||
// Current |
||||
$result = strlen($s); // returns mixed/int |
||||
|
||||
// After optimization |
||||
$result = _strlen_string($s); // compile-time-determined int return |
||||
``` |
||||
|
||||
--- |
||||
|
||||
### 2.7 Class Assumptions: A Priori Type Prediction |
||||
|
||||
**File**: `compiler/class-assumptions.cpp` |
||||
|
||||
Solves the **circular dependency between type inference and call graph construction**: |
||||
|
||||
``` |
||||
$obj->method() needs $obj's type to bind method() |
||||
but type inference needs a complete call graph |
||||
→ Assumption breaks the cycle |
||||
``` |
||||
|
||||
Assumption sources: |
||||
- `@param ClassName $x` — parameter type |
||||
- `@return ClassName` — return type |
||||
- `@var ClassName` — local variable |
||||
- Constructor call `new ClassName()` → directly obtains the type |
||||
|
||||
Assumptions are made **before** type inference, used to bind the call graph. After type inference they are **validated** — an error is reported on mismatch. |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
For method call devirtualization: assumptions are available earlier than pure SSA analysis and can serve as the first stage of devirtualization (falling back when SSA is unavailable). |
||||
|
||||
--- |
||||
|
||||
### 2.8 Automatic Virtual Method Generation |
||||
|
||||
**File**: `compiler/pipes/generate-virtual-methods.cpp` |
||||
|
||||
When a method is overridden by subclasses, the base class method automatically becomes a dispatcher: |
||||
|
||||
```cpp |
||||
ReturnType f$Base$$method(instance_var, args...) { |
||||
if (instance_var.ce() == Child1::ce) |
||||
return f$Child1$$method(instance_cast<Child1>(instance_var), args...); |
||||
if (instance_var.ce() == Child2::ce) |
||||
return f$Child2$$method(instance_cast<Child2>(instance_var), args...); |
||||
// ... fallback to self |
||||
return f$Base$$method$$Base(instance_var, args...); |
||||
} |
||||
``` |
||||
|
||||
It also performs PHP 7.4+ type variance checks (parameter contravariance, return covariance). |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
The "runtime exact-type guard" in the current devirtualization plan is consistent with this idea. Its automatic generation of all dispatch branches + variance checking can be adopted. |
||||
|
||||
--- |
||||
|
||||
### 2.9 Performance Inspection Annotations |
||||
|
||||
**File**: `docs/kphp-language/best-practices/performance-inspections.md` |
||||
|
||||
Compile-time performance analysis, activated via annotations: |
||||
|
||||
```php |
||||
/** @kphp-warn-performance implicit-array-cast */ |
||||
function businessLogic() { ... } |
||||
``` |
||||
|
||||
Supported inspection items: |
||||
- `implicit-array-cast` — detect `array<int>` → `array<mixed>` implicit conversion (expensive copy) |
||||
- `array-merge-into` — detect merges that can be optimized via `array_merge_into` |
||||
- `array-reserve` — detect arrays that can be pre-sized |
||||
- `constant-execution-in-loop` — detect constant expressions inside loops |
||||
|
||||
Annotations propagate through the call chain to all reachable functions. |
||||
|
||||
**AOT adoption priority: P3** |
||||
|
||||
Similar to function coloring, this can serve as a compile-time static analysis plugin. `implicit-array-cast` has a huge performance impact on typed arrays and is worth detecting separately. |
||||
|
||||
--- |
||||
|
||||
### 2.10 Pooled Memory Allocator |
||||
|
||||
**File**: `runtime-common/core/memory-resource/unsynchronized_pool_resource.h` |
||||
|
||||
- Pre-allocates fixed-size buffers |
||||
- Small blocks (<16KB): slab allocation, graded by size (`free_chunks_[chunk_id]`), O(1) allocate/free |
||||
- Large blocks (≥16KB): red-black tree management (`huge_pieces_`), supporting defragmentation |
||||
- **Hard reset after each request** (`hard_reset()`), no per-object freeing needed |
||||
- Supports OOM handling memory reservation |
||||
|
||||
**AOT adoption priority: P4** |
||||
|
||||
Currently relies on Zend MM. For long-running CLI mode, a pool allocator can significantly reduce fragmentation and allocation overhead. But it requires replacing the entire memory management layer, a large effort. |
||||
|
||||
--- |
||||
|
||||
## 3. Toolchain Analysis |
||||
|
||||
### 3.1 Test Infrastructure |
||||
|
||||
KPHP has a three-layer test system: |
||||
|
||||
| Layer | Directory | Purpose | |
||||
|------|------|------| |
||||
| PHPT tests | `tests/phpt/` (75+ subdirectories) | PHP behavior compatibility tests | |
||||
| C++ unit tests | `tests/cpp/compiler/` `tests/cpp/runtime/` `tests/cpp/server/` | Compiler/runtime/server component tests | |
||||
| Python integration tests | `tests/python/tests/` | HTTP/RPC/multi-process integration tests | |
||||
|
||||
The test runner `tests/kphp_tester.py` supports: |
||||
- Tag mechanism (`@ok`, `@kphp_should_fail`, `@kphp_should_warn`, etc.) |
||||
- PHP version selection (`@php7.4`, `@php8`) |
||||
- Multi-process parallel execution (based on ThreadPool) |
||||
- TCP server management |
||||
- k2 mode (component compilation) compatibility |
||||
- Incremental compilation support (nocc distributed compilation) |
||||
|
||||
**AOT adoption**: |
||||
- Currently AOT only has two layers — `phpunit/` (PHPUnit) and `tests/compiler/` (PHPT) — lacking compiler internal unit tests and integration tests |
||||
- The tag mechanism is more flexible than pure PHPT — it can mark expected compile failures, expected warnings, etc. |
||||
- The Python test runner provides better CI integration capability |
||||
|
||||
### 3.2 Benchmark Framework |
||||
|
||||
**File**: `tests/benchmarks/` |
||||
|
||||
Uses the Go-written `ktest` tool for KPHP vs PHP performance comparison: |
||||
|
||||
``` |
||||
$ KPHP_ROOT=/path/to/repo/kphp ./ktest bench-vs-php tests/benchmarks/ |
||||
``` |
||||
|
||||
Benchmark coverage: |
||||
- `BenchmarkBasic.php` — basic operations |
||||
- `BenchmarkConcat.php` — string concatenation |
||||
- `BenchmarkExplode.php` — explode performance |
||||
- `BenchmarkMultiSwitch.php` — large switch |
||||
- `BenchmarkTmpString.php` — temporary string optimization effect |
||||
- `BenchmarkJson.php` / `BenchmarkFFI.php` — specific features |
||||
|
||||
**AOT adoption**: |
||||
- Can build a similar AOT vs PHP benchmark comparison suite |
||||
- Especially focus on scenarios the AOT compiler claims to optimize (such as typed property access, devirtualized calls) |
||||
|
||||
### 3.3 IDE Integration |
||||
|
||||
KPHP provides the **kphpstorm** IDE plugin (`docs/kphp-language/kphpstorm-ide-plugin/`), supporting: |
||||
- `@kphp-*` annotation syntax highlighting |
||||
- Type annotation completion |
||||
- Hints for KPHP-specific types |
||||
|
||||
**AOT adoption**: |
||||
- Currently the AOT compiler uses annotations like `@phpstan-*`; can consider providing VSCode/JetBrains plugins |
||||
|
||||
### 3.4 Incremental Compilation |
||||
|
||||
KPHP recompiles only changed files (based on CRC64 hashes): |
||||
|
||||
```cpp |
||||
// At the start of each generated file |
||||
//crc64 <content_hash> |
||||
//crc64_with_comments <hash_with_comments> |
||||
``` |
||||
|
||||
These hashes are compared against the previous generation results to determine which files need recompilation, including all upstream files that depend on them. |
||||
|
||||
**AOT adoption**: |
||||
- Currently the `build/` directory is fully regenerated; a similar incremental mechanism can be introduced to speed up iteration on large projects |
||||
|
||||
--- |
||||
|
||||
## 4. Syntax Compatibility Analysis |
||||
|
||||
### 4.1 PHP Versions Supported by KPHP |
||||
|
||||
KPHP targets the **PHP 7.4** language level, with some 8.0/8.1 features being added. |
||||
|
||||
### 4.2 Unsupported Features (Architectural Reasons) |
||||
|
||||
| Feature | Reason | |
||||
|------|------| |
||||
| Dynamic function/method calls (`call_user_func`) | Symbols cannot be resolved at compile time | |
||||
| `eval()` | Unknown at compile time | |
||||
| Dynamic class/function declarations | Symbol table must be complete at compile time | |
||||
| Reflection | Requires runtime metadata | |
||||
| Mock (PHPUnit) | Depends on Reflection + dynamic redefinition | |
||||
| Array internal pointers (`reset`/`current`/`next`) | Not consistent with reference semantics | |
||||
| PHP extension interop | Replaced by self-developed runtime | |
||||
|
||||
### 4.3 Unsupported Features (Not Implemented) |
||||
|
||||
| Feature | Status | |
||||
|------|------| |
||||
| Nested `list()` | Not implemented | |
||||
| Generators (`yield`) | Not implemented | |
||||
| Anonymous classes | Not implemented | |
||||
| Group use declarations | Not implemented | |
||||
| finally | Not implemented | |
||||
| `func_get_args` | Not implemented | |
||||
| References (except foreach by ref and reference parameters) | Partially supported | |
||||
| Interface appearing multiple times in a parent chain | Not supported | |
||||
| `insteadof` / trait renaming | Not supported | |
||||
|
||||
### 4.4 KPHP-Specific Annotations |
||||
|
||||
```php |
||||
// Function annotations |
||||
@kphp-inline // force inlining (GCC inline) |
||||
@kphp-flatten // aggressively inline all callees |
||||
@kphp-required // force compilation (for string callbacks) |
||||
@kphp-sync // forbid being resumable |
||||
@kphp-no-return // never returns (optimizes CFG) |
||||
@kphp-pure-function // pure function (callable on constant arrays) |
||||
@kphp-warn-unused-result // error on unused return value |
||||
@kphp-should-not-throw // forbid throwing exceptions |
||||
@kphp-throws {Class} // checked exceptions |
||||
@kphp-generic T1, T2 // generic functions |
||||
@kphp-color {color} // capability annotation |
||||
@kphp-warn-performance {...} // performance inspection |
||||
@kphp-disable-warnings {...} // suppress specific warnings |
||||
@kphp-profile // embed profiler |
||||
|
||||
// Class annotations |
||||
@kphp-serializable // serializable |
||||
@kphp-immutable-class // immutable class |
||||
@kphp-json {attr}={value} // JSON configuration |
||||
``` |
||||
|
||||
### 4.5 Syntax Differences from the AOT Compiler |
||||
|
||||
| Feature | KPHP | AOT Compiler | |
||||
|------|------|-------------| |
||||
| Base PHP version | 7.4 | 8.2+ | |
||||
| Enums | Not supported | Supported (PHP 8.1 enum) | |
||||
| Named arguments | Not supported | Supported | |
||||
| Match expressions | Not supported | Supported | |
||||
| Union types | Partial support | Supported | |
||||
| Nullsafe `?->` | Not supported | Supported | |
||||
| Constructor promotion | Not supported | Supported | |
||||
| `list()` destructuring | Partial | Full support | |
||||
| `break N` / `continue N` | Partial support | Supported | |
||||
| Typed arrays | Custom syntax `array<T>` | None (phpstan annotations) | |
||||
| Generic functions | `@kphp-generic` | None | |
||||
| Tuples / Shapes | Custom syntax | None | |
||||
| FFI | Supported (custom FFI) | None | |
||||
|
||||
### 4.6 Key Differences in the Type System |
||||
|
||||
KPHP's type system is much stricter than PHP's: |
||||
- **No type mixing allowed**: `f(42); f("string")` for the same `$arg` is a compile error |
||||
- **Typed arrays**: `array<int>` vs `array<string>` are different types; conversion requires explicit or implicit cast |
||||
- **mixed is expensive**: 16-byte tagged union + switch-case dispatch |
||||
- **Generic functions**: implemented via `@kphp-generic` compile-time specialization, similar to C++ templates |
||||
- **Variable splitting**: the same variable name may split into different names on different CFG paths (e.g. `$x` → `$x$v1`) |
||||
|
||||
--- |
||||
|
||||
## 5. Summary and Priority Recommendations |
||||
|
||||
| Priority | Technique | Difficulty | Benefit | Notes | |
||||
|--------|------|------|------|------| |
||||
| **P0** | Rewrite Rules DSL | Medium | High | Declarative optimization rules, extremely extensible | |
||||
| **P0** | Smart instanceof casts | Low | High | Directly improves object type narrowing + devirtualization | |
||||
| **P1** | Function specialization (multi-version) | Medium | High | More precise return types, eliminates mixed pollution | |
||||
| **P1** | Immutable constant markers | Low | Medium | Reduces COW; constant promotion foundation already exists | |
||||
| **P2** | Switch splitting | Medium | Medium | Multi-level break foundation already exists; optimization for specific scenarios | |
||||
| **P2** | Class Assumptions | High | High | Requires PHPDoc parsing infrastructure | |
||||
| **P2** | Automatic virtual method generation | Medium | High | Complements the devirtualization plan | |
||||
| **P3** | Performance inspection annotations | Low | Medium | Helps developers discover hidden performance issues | |
||||
| **P3** | Pipeline parallelism | High | Medium | Faster compilation of large projects | |
||||
| **P3** | Function coloring | Low | Low | Aids security/IO auditing | |
||||
| **P4** | Incremental compilation | Medium | Medium | Development experience optimization | |
||||
| **P5** | Resumable state machine | High | Medium | Requires actual async needs | |
||||
| **P5** | Pool allocator | Very high | High | Requires replacing the entire memory management | |
||||
@ -0,0 +1,525 @@ |
||||
# PeachPie Compiler Review: Roslyn Integration, Cross-Language Interop, and Type System |
||||
|
||||
> 2026-06-11 · Based on source review of /home/swoole/workspace/cpp/peachpie |
||||
|
||||
--- |
||||
|
||||
## 1. Architecture Overview |
||||
|
||||
PeachPie is a **PHP-to-.NET compiler based on Roslyn (Microsoft's .NET compiler platform)**, with roughly 710 C# source files and 272k lines of code. |
||||
|
||||
### Compilation Pipeline |
||||
|
||||
``` |
||||
PHP source |
||||
→ PhpSyntaxTree (Roslyn SyntaxTree for PHP — Syntax/) |
||||
→ SemanticModel + Symbols (Roslyn symbol system — Semantics/, Symbols/) |
||||
→ BoundControlFlowGraph (CFG with typed IR — Semantics/Graph/, FlowAnalysis/) |
||||
→ CIL Bytecode (EMIT — CodeGen/, Emitter/) |
||||
→ .NET Assembly (.dll / .exe) |
||||
``` |
||||
|
||||
| Stage | Component | Responsibility | |
||||
|------|------|------| |
||||
| Syntax parsing | `Syntax/` (PhpSyntaxTree, NodesFactory) | PHP → Roslyn SyntaxTree | |
||||
| Semantic binding | `Semantics/` (SemanticsBinder, BoundExpression) | Name resolution, method binding, type inference | |
||||
| Symbol system | `Symbols/` (SourceTypeSymbol, PEMethodSymbol...) | Roslyn symbol table for types/methods/properties | |
||||
| Dataflow analysis | `FlowAnalysis/` (FlowState, TypeRefMask, ExpressionAnalysis) | Type inference, unreachable code detection, conditional narrowing | |
||||
| CFG optimization | `FlowAnalysis/Passes/` (TransformationRewriter) | CFG rewriting, constantization, dead code elimination | |
||||
| Code generation | `CodeGen/` (CodeGenerator, GhostMethodBuilder) | IR → CIL instructions | |
||||
| Assembly output | `Emitter/` (PEModuleBuilder) | CIL → PE file (.dll/.exe) | |
||||
|
||||
### Core Files |
||||
|
||||
| File | Lines | Purpose | |
||||
|------|------|------| |
||||
| `CodeGen/Graph/BoundExpression.cs` | 5,742 | Core IR node definitions and CIL emission | |
||||
| `CodeGen/CodeGenerator.Emit.cs` | 4,396 | CIL instruction generation | |
||||
| `FlowAnalysis/ExpressionAnalysis.cs` | 2,911 | Expression-level type analysis | |
||||
| `Semantics/BoundExpression.cs` | 2,721 | Semantic binding expressions | |
||||
| `Runtime/Operators.cs` | 2,573 | Runtime implementation of PHP operators | |
||||
| `Runtime/PhpString.cs` | 2,047 | PHP string value type | |
||||
| `CodeGen/VariableReference.cs` | 1,863 | Variable references and address analysis | |
||||
| `Symbols/Source/SourceTypeSymbol.cs` | 1,795 | Source type symbols | |
||||
| `Runtime/Conversions.cs` | 1,603 | Type conversion runtime | |
||||
|
||||
### Comparison with AOT Compiler |
||||
|
||||
| Dimension | PeachPie | AOT Compiler | |
||||
|------|----------|-------------| |
||||
| Compilation target | PHP → CIL → .NET Assembly | PHP → C++ → binary | |
||||
| Compiler framework | Roslyn (C# compiler-as-a-library) | Self-built PHP AST → C++ string | |
||||
| IR form | BoundControlFlowGraph (Roslyn pattern) | PHP-Parser AST nodes | |
||||
| Type inference | FlowState + TypeRefMask bitset | SSA + manual type annotations | |
||||
| Symbol system | Roslyn Symbol hierarchy (complete) | Simplified ClassDef/FunctionDef | |
||||
| Output | .NET PE file (cross-platform) | Native binary (Linux/Mac/Windows) | |
||||
| Runtime | Peachpie.Runtime (PhpValue, PhpArray, PhpString) | phpx (C++ RAII wrapper around Zend API) | |
||||
| Cross-language interop | First-class citizen — PHP ⇄ C# bidirectional calls | FFI only (swoole_cc/cpp extension loading) | |
||||
| Parallel compilation | Parallel.ForEach function-level parallelism | Serial execution | |
||||
| Code size | ~272k lines of C# (including runtime) | ~15k lines of PHP | |
||||
| MSBuild integration | Complete SDK (`dotnet build`) | None | |
||||
|
||||
--- |
||||
|
||||
## 2. Reusable Innovative Designs |
||||
|
||||
### 2.1 Roslyn-Based Compiler Architecture |
||||
|
||||
**Files**: all of `Peachpie.CodeAnalysis/` |
||||
|
||||
PeachPie's most central design decision is to be **fully built on the Microsoft Roslyn compiler platform**. This means: |
||||
|
||||
- **Reuse Roslyn's symbol system**: `TypeSymbol`, `MethodSymbol`, `NamedTypeSymbol`, and other standard Roslyn types |
||||
- **Reuse Roslyn's metadata emission**: `PEModuleBuilder`, `PEAssemblyBuilder` directly generate PE files |
||||
- **Reuse Roslyn's diagnostics system**: `DiagnosticBag`, standardized Error/Warning mechanism |
||||
- **Native MSBuild integration**: PHP projects are standard .NET projects (`.csproj` style), compilable with `dotnet build` |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
Currently AOT built all compiler infrastructure from scratch. PeachPie's approach is a useful reference, but not suitable for direct porting (AOT's goal is to generate C++ rather than CIL). What can be adopted: |
||||
|
||||
- Decompose the compiler into standard stages — **Syntax → Semantic → IR → CodeGen** — with clear interfaces per stage |
||||
- Separate Preprocessor (symbol collection) into an independent Analyzer stage, similar to Roslyn's Compilation concept |
||||
- Define a unified `Diagnostic` type instead of scattered `fatalError()` / `SyntaxError` calls |
||||
|
||||
--- |
||||
|
||||
### 2.2 TypeRefMask: 64-bit Type Bitset |
||||
|
||||
**File**: `FlowAnalysis/TypeRef/TypeRefMask.cs` |
||||
|
||||
PeachPie uses `ulong` (64-bit) as the type mask: |
||||
|
||||
```csharp |
||||
public struct TypeRefMask { |
||||
ulong _mask; |
||||
// bits 0-61: type index (up to 62 different types) |
||||
// bit 62: IncludesSubclasses (type may include subclasses) |
||||
// bit 63: IsRef (value is a reference/alias) |
||||
} |
||||
``` |
||||
|
||||
Characteristics: |
||||
- **O(1) type comparison**: `(mask & type_bit) != 0` detects a type |
||||
- **Union types**: `mask1 | mask2` = contains both types |
||||
- **Type narrowing**: `mask & ~excluded_type_bit` = excludes a type |
||||
- **IsRef marker**: tracks whether a value is assigned by reference, affecting alias analysis |
||||
- **IncludesSubclasses**: distinguishes `exactly Class` vs `Class or subclass` |
||||
|
||||
Each `TypeRefContext` maintains a type registry mapping concrete .NET types (such as `System.Int64`, `Pchp.Core.PhpString`) to bit indices. |
||||
|
||||
**AOT adoption priority: P0** |
||||
|
||||
This is consistent with HHVM's trep idea — using a bitset to represent types. AOT's discrete constants like `TYPE_INT`, `TYPE_STRING` can be replaced with a bitset: |
||||
|
||||
```php |
||||
class TypeMask { |
||||
const BINT = 1 << 0; |
||||
const BFLOAT = 1 << 1; |
||||
const BSTRING = 1 << 2; |
||||
const BBOOL = 1 << 3; |
||||
const BARRAY = 1 << 4; |
||||
const BOBJECT = 1 << 5; |
||||
// Extension markers |
||||
const BEMPTY = 1 << 60; // empty value marker |
||||
const BSUBCLASS = 1 << 61; // allow subclasses |
||||
const BREFERENCE = 1 << 62; // reference marker |
||||
} |
||||
``` |
||||
|
||||
Advantage: `int|string` = `BINT|BSTRING`, type narrowing = `& ~excluded_bits`. |
||||
|
||||
--- |
||||
|
||||
### 2.3 FlowState + Worklist Dataflow Analysis |
||||
|
||||
**Files**: `FlowAnalysis/FlowState.cs`, `FlowAnalysis/Worklist.cs` |
||||
|
||||
PeachPie uses the classic **dataflow worklist algorithm** for type inference: |
||||
|
||||
```csharp |
||||
class FlowState { |
||||
TypeRefMask[] _varsType; // type mask for each variable |
||||
ulong _initializedMask; // whether variables are initialized |
||||
HashSet<NoteData> _notes; // additional info (such as function return points) |
||||
} |
||||
``` |
||||
|
||||
**Merge operation**: when two CFG paths converge, the `FlowState(state1, state2)` constructor computes: |
||||
- The **union** of type masks (all possible types) |
||||
- The **union** of initialization masks (initialized in either branch counts as initialized) |
||||
- The **intersection** of notes (only info present on both paths is retained) |
||||
|
||||
The Worklist processes blocks in topological order, re-enqueueing when state changes. |
||||
|
||||
**AOT adoption priority: P1** |
||||
|
||||
Currently AOT's SSA analysis does some type inference, but lacks: |
||||
- **Structured FlowState**: a unified variable type-state representation |
||||
- **Standard merge operation**: type merging at join points |
||||
- **Worklist iteration**: an iterative analysis framework reaching a fixed point |
||||
|
||||
--- |
||||
|
||||
### 2.4 ConditionBranch-Aware Type Narrowing |
||||
|
||||
**Files**: `FlowAnalysis/ConditionBranch.cs`, `FlowAnalysis/AnalysisFacts.cs` |
||||
|
||||
PeachPie's type analysis is **aware of the conditional branch of the current context**: |
||||
|
||||
```csharp |
||||
enum ConditionBranch { |
||||
AnyResult = 0, // ordinary evaluation |
||||
ToTrue = +1, // branch where the expression result is true |
||||
ToFalse = -1, // branch where the expression result is false |
||||
} |
||||
``` |
||||
|
||||
In conditional expressions, the analyzer carries the branch direction to propagate type info: |
||||
|
||||
```csharp |
||||
// if ($x instanceof MyClass) { ... } |
||||
// In the ToTrue branch: |
||||
// $x's type is narrowed, excluding types that cannot be MyClass |
||||
// In the ToFalse branch: |
||||
// $x's type is narrowed, excluding MyClass |
||||
|
||||
// if (is_int($x)) { ... } |
||||
// In the ToTrue branch: |
||||
// $x's type is narrowed to int |
||||
``` |
||||
|
||||
`AnalysisFacts.HandleSpecialFunctionCall()` registers type-check functions like `is_int`, `is_string`, `is_array`, `is_callable`, `function_exists`, `class_exists`, automatically narrowing variable types in branches. |
||||
|
||||
**AOT adoption priority: P1** |
||||
|
||||
Currently `SsaTypeOptimizer` does some instanceof narrowing, but: |
||||
- Does not support narrowing via built-in type-check functions like `is_int()` / `is_string()` |
||||
- Does not support constant folding of existence checks like `class_exists()` / `function_exists()` |
||||
- Can directly adopt `AnalysisFacts`'s "known type-check function registry" pattern |
||||
|
||||
--- |
||||
|
||||
### 2.5 PhpValue Tagged Union Design |
||||
|
||||
**File**: `Peachpie.Runtime/PhpValue.cs` |
||||
|
||||
PeachPie's runtime value type uses an ingenious C# tagged union: |
||||
|
||||
```csharp |
||||
[StructLayout(LayoutKind.Sequential)] |
||||
public readonly partial struct PhpValue { |
||||
readonly PhpTypeCode _type; // 1-byte type tag |
||||
|
||||
// Explicit-layout union: two fields occupy the same memory |
||||
[StructLayout(LayoutKind.Explicit)] |
||||
struct ValueField { |
||||
[FieldOffset(0)] public bool @bool; |
||||
[FieldOffset(0)] public long @long; |
||||
[FieldOffset(0)] public double @double; |
||||
} |
||||
|
||||
[StructLayout(LayoutKind.Explicit)] |
||||
struct ObjectField { |
||||
[FieldOffset(0)] public object @object; |
||||
[FieldOffset(0)] public string @string; |
||||
[FieldOffset(0)] public PhpString.Blob blob; |
||||
[FieldOffset(0)] public PhpArray array; |
||||
[FieldOffset(0)] public PhpAlias alias; |
||||
} |
||||
|
||||
readonly ValueField _value; // value-type storage |
||||
readonly ObjectField _obj; // reference-type storage |
||||
} |
||||
``` |
||||
|
||||
Memory layout: `PhpValue` = `PhpTypeCode` (1 byte) + padding + `ValueField` (8 bytes) + `ObjectField` (8 bytes, pointer) ≈ 24 bytes. |
||||
|
||||
Advantages of this design: |
||||
- **readonly struct** — no GC overhead, allocatable on the stack |
||||
- **Explicit union** — value types and reference types share space, compact |
||||
- **PhpAlias mechanism** — abstracts PHP references (copy-on-write) via `PhpAlias` rather than directly copying values |
||||
- **MutableString** — distinguishes immutable string from writable MutableString (for string concatenation optimization) |
||||
|
||||
**AOT adoption priority: P3** |
||||
|
||||
Currently AOT uses `Variant` (based on Zend `zval`) as the dynamic type. What can be adopted: |
||||
- PhpString's MutableString separation (string builder pattern) |
||||
- PhpAlias's reference semantics abstraction |
||||
- But replacing `Variant` entirely is a huge effort, so low priority |
||||
|
||||
--- |
||||
|
||||
### 2.6 GhostMethodBuilder: PHP Method ⇄ C# Method Adaptation |
||||
|
||||
**File**: `CodeGen/GhostMethodBuilder.cs` |
||||
|
||||
PeachPie's most distinctive feature is **automatically generating ghost stub methods** so PHP methods can be called directly from C#: |
||||
|
||||
```csharp |
||||
// Generate C#-callable wrappers for PHP methods: |
||||
// - Handle parameter type conversion (PhpValue → CLR type) |
||||
// - Handle return type conversion (CLR type → PhpValue) |
||||
// - Build and pass PhpContext |
||||
// - Support explicit interface override |
||||
static MethodSymbol CreateGhostOverload( |
||||
MethodSymbol original, NamedTypeSymbol containingtype, |
||||
PEModuleBuilder module, DiagnosticBag diagnostic, |
||||
TypeSymbol ghostreturn, ImmutableArray<ParameterSymbol> ghostparams, |
||||
bool phphidden = false, MethodSymbol explicitOverride = null) |
||||
``` |
||||
|
||||
Ghost methods enable: |
||||
- Type-safe interfaces when C# calls PHP methods |
||||
- PHP implementing C# interfaces (IMethod, INotifyPropertyChanged, etc.) |
||||
- PHP classes usable as .NET generic parameters |
||||
|
||||
**AOT adoption priority: P4** |
||||
|
||||
Currently AOT does not support PHP calling C++ (or vice versa). If a bidirectional interop layer is needed in the future, the ghost stub pattern is worth referencing. |
||||
|
||||
--- |
||||
|
||||
### 2.7 DelayedTransformations: Parallel-Safe Deferred Transformations |
||||
|
||||
**File**: `FlowAnalysis/Passes/DelayedTransformations.cs` |
||||
|
||||
During the parallel analysis phase, certain transformations (such as marking unreachable functions, promoting conditional functions to unconditional ones) cannot directly modify shared state. PeachPie uses the **deferred transformation** pattern: |
||||
|
||||
```csharp |
||||
class DelayedTransformations { |
||||
ConcurrentBag<SourceRoutineSymbol> UnreachableRoutines; |
||||
ConcurrentBag<SourceTypeSymbol> UnreachableTypes; |
||||
ConcurrentBag<SourceFunctionSymbol> FunctionsMarkedAsUnconditional; |
||||
// Collected thread-safely during parallel analysis |
||||
// Applied serially via Apply() after analysis completes |
||||
} |
||||
``` |
||||
|
||||
Analysis threads only place the objects to be transformed into the `ConcurrentBag`, and after analysis completes a single thread calls `Apply()`. |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
AOT is currently serial and doesn't need this. But if parallel compilation is introduced in the future (see KPHP review 2.3), deferred transformation is the foundational pattern for thread safety. |
||||
|
||||
--- |
||||
|
||||
### 2.8 Native MSBuild Integration (Peachpie.NET.Sdk) |
||||
|
||||
**File**: `Peachpie.NET.Sdk/` |
||||
|
||||
PeachPie is not just a compiler — it is a **complete .NET SDK**: |
||||
|
||||
``` |
||||
dotnet new classlibrary -o MyPhpLib # create a PHP class library project |
||||
dotnet build # compile PHP → .NET DLL |
||||
dotnet run # run the compiled program |
||||
dotnet publish # publish as a self-contained app |
||||
``` |
||||
|
||||
Implemented via MSBuild targets/props: |
||||
- `build/peachpie.targets` — defines the compilation task |
||||
- `Peachpie.NET.Sdk.nuspec` — NuGet package definition |
||||
- `BuildTask.cs` — MSBuild compilation task |
||||
|
||||
This means PHP projects can seamlessly use the .NET ecosystem: NuGet package references, project references, conditional compilation, multi-targeting, etc. |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
AOT currently uses `php bin/tpc.php <project>` on the command line. What can be adopted: |
||||
- Create a Composer plugin or CLI phar package for the AOT compiler |
||||
- Define a JSON Schema for `project.yml` (similar to `.csproj`) |
||||
- Support a unified entry point like `composer build` or `php-aot build` |
||||
|
||||
--- |
||||
|
||||
### 2.9 Lazily Evaluable AnalysisFacts |
||||
|
||||
**File**: `FlowAnalysis/AnalysisFacts.cs` |
||||
|
||||
PeachPie performs constant evaluation of a large number of PHP runtime functions at compile time: |
||||
|
||||
| Function | Evaluation strategy | |
||||
|------|---------| |
||||
| `function_exists(X)` | Check if symbol X exists in the PE assembly → fold to `true` | |
||||
| `class_exists(X)` | Check if type X exists in the PE assembly → fold to `true`/`false` | |
||||
| `method_exists(X, M)` | Check if method M exists in type X → fold to `true`/`false` | |
||||
| `defined(CONST)` | Check if the constant exists → fold to `true`/`false` | |
||||
| `is_callable(F)` | Check if F is unconditionally declared → fold to `true` | |
||||
| `dirname(__FILE__)` | Compile-time path computation → `__DIR__` | |
||||
| `basename(__FILE__)` | Compile-time filename extraction → string constant | |
||||
|
||||
These evaluations leverage PeachPie's "PE assembly" concept — already-compiled .NET assemblies contain complete type/method/constant metadata that can be **queried at compile time**. |
||||
|
||||
**AOT adoption priority: P1** |
||||
|
||||
Currently `FuncCallOptimizer` only does the most basic constant folding (`strlen("abc")` → `3`). It can be extended to compile-time evaluation of reflective functions like `function_exists`, `class_exists`, `defined` — provided a complete symbol table is built in Preprocessor. |
||||
|
||||
--- |
||||
|
||||
### 2.10 Conditional Declaration Detection and Unreachable Code Elimination |
||||
|
||||
**Files**: `FlowAnalysis/Passes/DelayedTransformations.cs`, `FlowAnalysis/Passes/TransformationRewriter.cs` |
||||
|
||||
PeachPie can detect **conditional declarations** (`if (condition) { function foo() {} }`) and optimize them: |
||||
|
||||
- If analysis proves the condition is always true → the function is marked as unconditionally declared |
||||
- If analysis proves the condition is always false → the function/class is marked as Unreachable and not compiled |
||||
|
||||
This enables writing C-like conditional compilation patterns in PHP: |
||||
|
||||
```php |
||||
if (PHP_VERSION_ID >= 80000) { |
||||
function newFeature() { ... } // not compiled on lower versions |
||||
} |
||||
``` |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
Currently AOT compiles every scanned function regardless of reachability. For projects with multi-version PHP compatibility code, conditional declaration elimination can reduce compilation artifact size. |
||||
|
||||
--- |
||||
|
||||
## 3. Toolchain Analysis |
||||
|
||||
### 3.1 Test Infrastructure |
||||
|
||||
PeachPie has 529 test files (PHP files) distributed across functional directories: |
||||
|
||||
| Directory | Content | |
||||
|------|------| |
||||
| `tests/arrays/` | Array operation tests (including a `lazy_copy` subdirectory) | |
||||
| `tests/classes/` | Class/object tests | |
||||
| `tests/functions/` | Function call tests | |
||||
| `tests/generators/` | Generator/yield tests | |
||||
| `tests/strings/` | String operation tests | |
||||
| `tests/operators/` | Operator tests | |
||||
| `tests/transformations/` | Compiler transformation/optimization tests | |
||||
| `tests/constants/` | Constant tests | |
||||
| `tests/constructs/` | Language construct tests | |
||||
| `tests/traits/` | Trait tests | |
||||
| `tests/reflection/` | Reflection tests | |
||||
| `tests/spl/` | SPL tests | |
||||
| `tests/bcmath/` `tests/hash/` `tests/pcre/` | Extension tests | |
||||
| `tests/pdo/` `tests/ftp/` `tests/openssl/` | Database/network extensions | |
||||
| `tests/gd/` `tests/xml/` `tests/zip/` | Graphics/XML/ZIP extensions | |
||||
| `tests/web/` `tests/scripting/` | Web/scripting integration tests | |
||||
|
||||
Test execution: compiled assemblies are executed through the .NET test framework (xUnit/NUnit). |
||||
|
||||
**AOT adoption:** |
||||
- The `tests/transformations/` directory specifically tests compiler optimizations — AOT can build similar optimization-correctness tests |
||||
- `tests/arrays/lazy_copy/` specifically tests copy-on-write behavior — AOT can also build dedicated COW tests |
||||
|
||||
### 3.2 Deep Visual Studio Integration |
||||
|
||||
PeachPie provides a complete IDE experience: |
||||
- **Visual Studio Extension** — project management, IntelliSense, debugging, performance analysis |
||||
- **VS Code / Rider support** — via OmniSharp / LSP |
||||
- **NuGet package management** — PHP libraries can be published and referenced as NuGet packages |
||||
|
||||
**AOT adoption:** |
||||
- Can provide a VSCode extension (Task integration + project templates) |
||||
- AOT's current `project.yml` can use a JSON Schema to provide IDE auto-completion |
||||
|
||||
### 3.3 Command-Line Toolchain |
||||
|
||||
```bash |
||||
# PeachPie's CLI experience |
||||
dotnet peach build # compile a PHP project |
||||
dotnet peach run # run the compiled program |
||||
dotnet peach publish # self-contained publishing |
||||
dotnet peach add <package> # add a dependency |
||||
``` |
||||
|
||||
**AOT adoption:** |
||||
```bash |
||||
# A similar CLI could be designed |
||||
php-aot build # compile a project |
||||
php-aot run # compile and run |
||||
php-aot new <name> # create a new project |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 4. Type System and Interop Analysis |
||||
|
||||
### 4.1 PeachPie Type Mapping |
||||
|
||||
| PHP type | PeachPie runtime type | .NET CLR type | |
||||
|----------|-------------------|---------------| |
||||
| null | `PhpTypeCode.Null` | `null` (any reference type) | |
||||
| bool | `PhpTypeCode.Boolean` | `bool` | |
||||
| int | `PhpTypeCode.Long` | `long` | |
||||
| float | `PhpTypeCode.Double` | `double` | |
||||
| string | `PhpTypeCode.String` / `MutableString` | `string` / `PhpString` | |
||||
| array | `PhpTypeCode.PhpArray` | `PhpArray` | |
||||
| object | `PhpTypeCode.Object` | `object` (concrete class) | |
||||
| reference | `PhpTypeCode.Alias` | `PhpAlias` | |
||||
|
||||
### 4.2 PHP ⇄ C# Interop |
||||
|
||||
PeachPie's bidirectional interop is its most prominent differentiator: |
||||
|
||||
**C# calling PHP:** |
||||
```csharp |
||||
// Compiled PHP classes become .NET classes; C# can directly new and call them |
||||
var phpObj = new MyPhpClass(ctx); |
||||
phpObj.someMethod(arg1, arg2); |
||||
``` |
||||
|
||||
**PHP calling C#:** |
||||
```php |
||||
// .NET types can be used directly in PHP |
||||
$list = new \System\Collections\Generic\List<int>; |
||||
$list->Add(42); |
||||
``` |
||||
|
||||
Interop implementation relies on: |
||||
- `GhostMethodBuilder` generating adapter methods |
||||
- `ConversionsExtensions` handling automatic PhpValue ↔ CLR type conversion |
||||
- `DynamicOperationFactory` handling dynamic method call forwarding |
||||
|
||||
### 4.3 Syntax Differences from the AOT Compiler |
||||
|
||||
| Feature | PeachPie | AOT Compiler | |
||||
|------|----------|-------------| |
||||
| Base PHP version | 8.0+ (target) | 8.2+ | |
||||
| Type annotations | Optional (gradually enriched) | Optional (phpstan annotations) | |
||||
| Namespaces | Standard PHP | Standard PHP | |
||||
| Generics | None | None | |
||||
| C# interop | Complete (first-class citizen) | FFI extension only | |
||||
| .NET ecosystem | Fully compatible | Not relevant | |
||||
| MSBuild integration | Complete | None | |
||||
| Reflection | Partial support | Not supported | |
||||
| yield/generator | Supported | Supported | |
||||
|
||||
--- |
||||
|
||||
## 5. Summary and Priority Recommendations |
||||
|
||||
| Priority | Technique | Difficulty | Benefit | Notes | |
||||
|--------|------|------|------|------| |
||||
| **P0** | TypeRefMask bitset type system | Medium | Very high | Union types, type narrowing, non-null markers — foundation for all optimization passes | |
||||
| **P1** | ConditionBranch type narrowing | Low | High | Automatic narrowing via `is_int()`/`is_string()` check functions, not covered by current SSA | |
||||
| **P1** | AnalysisFacts compile-time evaluation | Medium | High | Compile-time folding of `function_exists`/`class_exists`/`defined` | |
||||
| **P1** | FlowState worklist analysis framework | Medium | High | Structured variable type state, standard merge operation | |
||||
| **P2** | Roslyn-style compiler layering | High | Medium | Clear Syntax→Semantic→IR→CodeGen layering; current Preprocessor/CompilerBase responsibilities are mixed | |
||||
| **P2** | Conditional declaration detection | Low | Medium | Eliminate unreachable functions/classes, reduce compilation artifact size | |
||||
| **P2** | MSBuild integration / CLI unification | Low | Medium | Standardized project configuration, CI-friendly | |
||||
| **P3** | PhpValue tagged union | High | Medium | Compact memory layout, but replacing Variant/zval is a large effort | |
||||
| **P3** | DelayedTransformations | Low | Low | Only meaningful under parallel compilation | |
||||
| **P4** | GhostMethodBuilder interop | Very high | Low | Requires .NET or a similar FFI runtime | |
||||
| **P4** | Deep IDE integration | High | Low | Limited ROI for a VSCode extension | |
||||
|
||||
### Key Takeaways |
||||
|
||||
1. **PeachPie's biggest advantage is .NET ecosystem integration** — this is a natural advantage from its architectural choice rather than any single technical innovation. AOT can adopt its "compiler SDK + standardized CLI" thinking without porting specific techniques. |
||||
|
||||
2. **TypeRefMask bitset is the most directly portable design** — representing types with a 64-bit bitset while supporting union types, subclass markers, and reference markers. This is a common feature of HHBBC's trep and KPHP's type analysis, indicating that the bitset type lattice is the best practice for PHP compilers. |
||||
|
||||
3. **The ConditionBranch pattern is a low-cost, high-benefit enhancement** — carrying "expected result" info in conditional branch analysis enables type-check functions to automatically narrow variable types. AOT's SSA analysis can adopt it immediately. |
||||
|
||||
4. **GhostMethodBuilder reveals a general interop pattern** — generating adapter/thunk methods to bridge the calling conventions of two languages. Although AOT targets C++ rather than .NET, if cross-language call needs arise (such as PHP calling C extensions), this pattern can be reused. |
||||
|
||||
5. **PeachPie's roughly 530 tests** are notably fewer than KPHP (75+ directories) and HHVM (14,675), indicating relatively lower maturity. But its way of categorizing tests by functional module and optimization type is worth referencing. |
||||
@ -0,0 +1,682 @@ |
||||
# PHP Zend Optimizer / OPcache / JIT Design Analysis: Mechanisms That Can Be Introduced into an AOT Compiler |
||||
|
||||
This document analyzes the implementation of the Zend Optimizer, OPcache, and JIT in php-src (v8.4.14), identifying the design patterns, algorithms, and data-flow frameworks that can be introduced into an AOT compiler. |
||||
|
||||
Source locations: |
||||
- Optimizer: `~/soft/php/php-8.4.14/Zend/Optimizer/` |
||||
- OPcache: `~/soft/php/php-8.4.14/ext/opcache/` |
||||
- JIT: `~/soft/php/php-8.4.14/ext/opcache/jit/` |
||||
|
||||
--- |
||||
|
||||
## Overview |
||||
|
||||
PHP's optimizer is one of the most mature high-end optimizers for dynamic languages in the field. It contains complete compiler optimization infrastructure, including full SSA (Static Single Assignment) construction, e-SSA (Extended SSA) type/range inference, SCCP (Sparse Conditional Constant Propagation), escape analysis, dead code elimination, call graph analysis, profile-guided tracing JIT, and more. |
||||
|
||||
An AOT compiler can borrow from the following aspects: |
||||
|
||||
| Priority | Design/Module | Implementation Scale | Benefit | |
||||
|--------|----------|---------|------| |
||||
| P1 | Pass Pipeline architecture | ~100 lines of framework | Clear architecture, pluggable, split into O0/O1/O2 | |
||||
| P1 | SSA + e-SSA construction | Medium | Foundation for all advanced optimizations | |
||||
| P1 | Type inference (Type & Range Inference) | Medium | Precise type deduction capability | |
||||
| P2 | SCDF generic data-flow framework | ~300 lines | Reusable for SCCP/type inference/optimization | |
||||
| P2 | SCCP constant propagation | Medium | Conditional constant folding + unreachable code elimination | |
||||
| P2 | DCE dead code elimination | ~400 lines | Worklist-driven precise DCE | |
||||
| P3 | Escape Analysis | ~500 lines | Stack allocation, reference count elimination | |
||||
| P3 | Call Graph | ~400 lines | Cross-function analysis, inlining decisions, dead code | |
||||
| P4 | JIT IR framework | Very heavy | Reference the abstraction levels of its IR design | |
||||
| P4 | OPcache persistence/File Cache | Light | Serialized IR cache after optimization | |
||||
|
||||
--- |
||||
|
||||
## 1. Pass Pipeline Architecture |
||||
|
||||
### Design |
||||
|
||||
PHP defines 16 optimization passes, each corresponding to a bitmask bit (`zend_optimizer.h:28-46`): |
||||
|
||||
```c |
||||
#define ZEND_OPTIMIZER_PASS_1 (1<<0) // Simple local optimization (constant replacement/folding) |
||||
#define ZEND_OPTIMIZER_PASS_2 (1<<1) // |
||||
#define ZEND_OPTIMIZER_PASS_3 (1<<2) // Jump optimization |
||||
#define ZEND_OPTIMIZER_PASS_4 (1<<3) // INIT_FCALL_BY_NAME -> DO_FCALL |
||||
#define ZEND_OPTIMIZER_PASS_5 (1<<4) // CFG optimization (block pass) |
||||
#define ZEND_OPTIMIZER_PASS_6 (1<<5) // DFA optimization (type/range inference → single function) |
||||
#define ZEND_OPTIMIZER_PASS_7 (1<<6) // CALL GRAPH optimization (cross-function analysis) |
||||
#define ZEND_OPTIMIZER_PASS_8 (1<<7) // SCCP (constant propagation) |
||||
#define ZEND_OPTIMIZER_PASS_9 (1<<8) // Temporary variable optimization |
||||
#define ZEND_OPTIMIZER_PASS_10 (1<<9) // NOP removal |
||||
#define ZEND_OPTIMIZER_PASS_11 (1<<10) // Merge identical constants |
||||
#define ZEND_OPTIMIZER_PASS_12 (1<<11) // Adjust stack usage |
||||
#define ZEND_OPTIMIZER_PASS_13 (1<<12) // Remove unused variables |
||||
#define ZEND_OPTIMIZER_PASS_14 (1<<13) // DCE (dead code elimination) |
||||
#define ZEND_OPTIMIZER_PASS_15 (1<<14) // Collect constants (unsafe) |
||||
#define ZEND_OPTIMIZER_PASS_16 (1<<15) // Function inlining |
||||
``` |
||||
|
||||
### Pipeline Scheduling |
||||
|
||||
The `zend_optimize()` function (`zend_optimizer.c:1067-1183`) executes each pass in order, with each pass only processing the results produced by already-executed passes: |
||||
|
||||
``` |
||||
pass1 (constant folding) → pass3 (jump optimization) → pass4 (function call optimization) |
||||
→ pass5 (CFG) → pass6 (DFA + type inference) → pass9 (temporary variables) |
||||
→ pass10 (NOP removal) → pass11 (constant merging) → pass13 (variable cleanup) → ... |
||||
``` |
||||
|
||||
When `PASS_6 + PASS_7` are enabled simultaneously, `zend_optimize_script()` takes a more complex call graph path: |
||||
``` |
||||
build_call_graph → zend_optimize (per-func) → analyze_call_graph |
||||
→ build call_map → dfa_analyze_op_array (per-func) |
||||
→ dfa_optimize_op_array (per-func, with call context) |
||||
→ pass9 → pass11 → pass13 → pass12 (stack adjust) → redo_pass_two |
||||
``` |
||||
|
||||
### Key Design Points |
||||
|
||||
**1. Bitmask switches + registered passes** |
||||
|
||||
Users can combine arbitrary passes by bitmask. It also supports `zend_optimizer_register_pass()` to register external passes (such as JIT's own optimization passes): |
||||
|
||||
```c |
||||
static struct { |
||||
zend_optimizer_pass_t pass[ZEND_OPTIMIZER_MAX_REGISTERED_PASSES]; |
||||
int last; |
||||
} zend_optimizer_registered_passes; |
||||
``` |
||||
|
||||
Registered passes execute after all built-in passes (`zend_optimizer_call_registered_passes`). |
||||
|
||||
**2. Optional dump output per pass** |
||||
|
||||
Controlled via `debug_level`, supporting output of the intermediate representation before/after any pass for debugging and performance analysis. |
||||
|
||||
**3. Dual-layer optimization: per-function and script-level** |
||||
|
||||
- `zend_optimize(op_array, ctx)` — single-function optimization, conservative (does not use cross-function information) |
||||
- `zend_optimize_script(script, ...)` — whole-script optimization with call graph, enabling cross-function optimization |
||||
|
||||
### AOT Takeaways |
||||
|
||||
The AOT compiler can define a similar pass pipeline: |
||||
|
||||
```php |
||||
enum AotPass: int { |
||||
case CONSTANT_FOLD = 1 << 0; |
||||
case TYPE_CHECK_INSERT = 1 << 1; |
||||
case ESCAPE_ANALYSIS = 1 << 2; |
||||
case DEVIRTUALIZE = 1 << 3; |
||||
case DEAD_CODE_ELIM = 1 << 4; |
||||
case FUNCTION_INLINE = 1 << 5; |
||||
case LOOP_OPTIMIZE = 1 << 6; |
||||
case BOX_ALLOC_ELIM = 1 << 7; |
||||
} |
||||
``` |
||||
|
||||
Combined by optimization level: |
||||
|
||||
```php |
||||
const O0 = AotPass::TYPE_CHECK_INSERT->value; // Required baseline code generation |
||||
const O1 = O0 | AotPass::CONSTANT_FOLD->value; // Basic optimization |
||||
const O2 = O1 | AotPass::DEVIRTUALIZE->value | AotPass::FUNCTION_INLINE->value; |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 2. SSA (Static Single Assignment) + e-SSA |
||||
|
||||
### Data Structures |
||||
|
||||
SSA is built on top of the control flow graph (CFG): |
||||
|
||||
**CFG (`zend_cfg.h:84-92`):** |
||||
```c |
||||
typedef struct _zend_cfg { |
||||
int blocks_count; // Number of basic blocks |
||||
int edges_count; // Number of edges |
||||
zend_basic_block *blocks; // Array of basic blocks |
||||
int *predecessors; // Predecessor list |
||||
uint32_t *map; // opnum → block mapping |
||||
uint32_t flags; |
||||
} zend_cfg; |
||||
|
||||
typedef struct _zend_basic_block { |
||||
int *successors; // Successor block indices |
||||
uint32_t flags; |
||||
uint32_t start; // Starting opcode |
||||
uint32_t len; // Number of opcodes |
||||
int successors_count; |
||||
int predecessors_count; |
||||
int idom; // Immediate dominator |
||||
int loop_header; // Nearest loop header |
||||
int level; // Dominator tree depth |
||||
int children; // Linked list of dominated child blocks |
||||
} zend_basic_block; |
||||
``` |
||||
|
||||
**SSA (`zend_ssa.h:135-143`):** |
||||
```c |
||||
typedef struct _zend_ssa { |
||||
zend_cfg cfg; // Control flow graph |
||||
int vars_count; // Number of SSA variables |
||||
int sccs; // Number of strongly connected components |
||||
zend_ssa_block *blocks; // φ functions per basic block |
||||
zend_ssa_op *ops; // use-def information per instruction |
||||
zend_ssa_var *vars; // def-use chain per SSA variable |
||||
zend_ssa_var_info *var_info; // Type inference results (type bitmask + range) |
||||
} zend_ssa; |
||||
``` |
||||
|
||||
**SSA Op (`zend_ssa.h:82-92`):** |
||||
```c |
||||
typedef struct _zend_ssa_op { |
||||
int op1_use; |
||||
int op2_use; |
||||
int result_use; |
||||
int op1_def; // SSA variable defined by this instruction |
||||
int op2_def; |
||||
int result_def; |
||||
int op1_use_chain; // use-def chain |
||||
int op2_use_chain; |
||||
int res_use_chain; |
||||
} zend_ssa_op; |
||||
``` |
||||
|
||||
### e-SSA: Extended SSA with Pi Nodes |
||||
|
||||
This is one of the most elegant designs of the PHP optimizer. A Pi node is a special kind of φ function used to represent type/range constraints inferred from conditional branches. |
||||
|
||||
**Pi constraints (`zend_ssa.h:42-59`):** |
||||
```c |
||||
typedef struct _zend_ssa_range_constraint { |
||||
zend_ssa_range range; // Range constraint [min, max] |
||||
int min_var; // Symbolic lower-bound variable |
||||
int max_var; // Symbolic upper-bound variable |
||||
zend_ssa_negative_lat negative; // Negation potential |
||||
} zend_ssa_range_constraint; |
||||
|
||||
typedef struct _zend_ssa_type_constraint { |
||||
uint32_t type_mask; // Type mask (narrowed type after AND operation) |
||||
zend_class_entry *ce; // Class entry (for instanceof) |
||||
} zend_ssa_type_constraint; |
||||
|
||||
typedef union _zend_ssa_pi_constraint { |
||||
zend_ssa_range_constraint range; |
||||
zend_ssa_type_constraint type; |
||||
} zend_ssa_pi_constraint; |
||||
``` |
||||
|
||||
**How it works:** For a condition like `if ($x > 0)`: |
||||
- In the truthy branch, insert `Pi($x, range[1, LONG_MAX])` — constrain `$x`'s SSA variable to the range > 0 |
||||
- In the falsy branch, insert `Pi($x, range[LONG_MIN, 0])` — constrain `$x` to ≤ 0 |
||||
|
||||
This allows subsequent optimizations within a branch to use refined type/range information without altering the explicit assignment chain of the original variable. |
||||
|
||||
### SSA Construction Flow |
||||
|
||||
``` |
||||
1. zend_build_cfg() → Build the control flow graph (including dominator tree, loop detection) |
||||
2. zend_build_dfg() → Build the data flow graph (compute use/def sets) |
||||
3. zend_build_ssa() → Place φ functions → rename variables → build SSA form |
||||
4. zend_ssa_compute_use_def_chains() → Connect use-def chains |
||||
5. zend_ssa_find_sccs() → Find strongly connected components (for type inference) |
||||
6. zend_ssa_inference() → Type inference + range inference (populate var_info) |
||||
``` |
||||
|
||||
### AOT Takeaways |
||||
|
||||
The AOT compiler does not need SSA form (because it generates C++ code, not direct register manipulation), but the following e-SSA concepts can be used directly: |
||||
|
||||
1. **The Pi constraint concept:** Insert type-narrowing markers in conditional branches so that variables inside a branch have more precise types. This directly corresponds to the implementation basis of the TypeSpecifier / Type Narrowing (#7) mentioned earlier. |
||||
|
||||
2. **Type & Range information associated with every expression:** Similar to the SSA `var_info` design, AOT can maintain a `{type_mask, range, ce}` triple for every variable/expression in the FunctionContext. |
||||
|
||||
3. **Use-def chains for optimization decisions:** When determining whether a variable has exactly one `use`, SSA's use_chain provides O(1) lookup. |
||||
|
||||
--- |
||||
|
||||
## 3. Type Inference (Type & Range Inference) |
||||
|
||||
### The Type System |
||||
|
||||
PHP uses a bitmask to represent type information (defined in `zend_type_info.h`), which is its most distinctive design: |
||||
|
||||
```c |
||||
#define MAY_BE_UNDEF (1<< 0) |
||||
#define MAY_BE_NULL (1<< 1) |
||||
#define MAY_BE_FALSE (1<< 2) |
||||
#define MAY_BE_TRUE (1<< 3) |
||||
#define MAY_BE_LONG (1<< 4) |
||||
#define MAY_BE_DOUBLE (1<< 5) |
||||
#define MAY_BE_STRING (1<< 6) |
||||
#define MAY_BE_ARRAY (1<< 7) |
||||
#define MAY_BE_OBJECT (1<< 8) |
||||
#define MAY_BE_RESOURCE (1<< 9) |
||||
#define MAY_BE_REFERENCE (1<<10) |
||||
#define MAY_BE_CALLABLE (1<<11) |
||||
#define MAY_BE_ITERABLE (1<<12) |
||||
#define MAY_BE_VOID (1<<13) |
||||
#define MAY_BE_INDIRECT (1<<14) |
||||
|
||||
// Convenient combinations |
||||
#define MAY_BE_ANY (MAY_BE_NULL|MAY_BE_FALSE|MAY_BE_TRUE|...) |
||||
#define MAY_BE_TRUTHY (MAY_BE_TRUE|MAY_BE_LONG|... /* not 0/''/[]/null */) |
||||
#define MAY_BE_FALSEY (MAY_BE_UNDEF|MAY_BE_NULL|MAY_BE_FALSE|...) |
||||
``` |
||||
|
||||
**Core advantage:** Bit operations are extremely fast. Type operations (merge/intersection/difference) require only a single AND/OR/NOT instruction: |
||||
|
||||
```c |
||||
// Merge the types of two variables |
||||
uint32_t result_type = info1 | info2; |
||||
|
||||
// Check whether it might be a string |
||||
if (info & MAY_BE_STRING) { ... } |
||||
|
||||
// Intersection |
||||
uint32_t common = info1 & info2; |
||||
``` |
||||
|
||||
### Range Inference |
||||
|
||||
Each SSA variable carries a `zend_ssa_range { min, max, underflow, overflow }`: |
||||
|
||||
```c |
||||
typedef struct _zend_ssa_range { |
||||
zend_long min; |
||||
zend_long max; |
||||
bool underflow; // Whether there is an underflow risk |
||||
bool overflow; // Whether there is an overflow risk |
||||
} zend_ssa_range; |
||||
``` |
||||
|
||||
The core algorithm (`zend_inference.c:1071`) is based on V. Campos's "Speed and Precision in Range Analysis, SBLP'12" paper: |
||||
|
||||
1. **Warmup phase (16 passes):** Propagate ranges over SCCs (strongly connected components), using widening to accelerate convergence |
||||
2. **Narrowing phase:** Gradually narrow the ranges, eliminating the over-approximation caused by widening |
||||
3. **Zend Engine-specific arithmetic semantics:** `zend_add_will_overflow()`, `zend_sub_will_overflow()` and others precisely detect integer overflow |
||||
|
||||
**Operator range inference example:** |
||||
|
||||
```c |
||||
// ADD: result range |
||||
min = OP1_MIN() + OP2_MIN() |
||||
max = OP1_MAX() + OP2_MAX() |
||||
overflow = OP1_RANGE_OVERFLOW() || OP2_RANGE_OVERFLOW() |
||||
|| zend_add_will_overflow(OP1_MAX(), OP2_MAX()) |
||||
|
||||
// Result type: if overflow is true, add MAY_BE_DOUBLE to the type |
||||
// (PHP int overflow automatically converts to float) |
||||
``` |
||||
|
||||
### `update_type_info` per Opcode |
||||
|
||||
`_zend_update_type_info()` is a huge switch that precisely computes the result type and range for each Zend opcode. For example, `ZEND_ASSIGN_DIM` (array assignment) updates not only the type of the assigned element, but also the type of the array as a whole, considering MAY_BE_PACKED_GUARD (packed array guard) and reference count inference. |
||||
|
||||
### AOT Takeaways |
||||
|
||||
1. **Bitmask type system:** It is the most suitable lightweight type representation for the AOT compiler. AOT currently uses string types (`TYPE_INT = 'int'`), which cannot efficiently represent compound types like "may be int or string". Bitmasks provide O(1) union/intersect/test operations. |
||||
|
||||
2. **Range inference:** Can select the optimal integer type for C++ code generation (`int32_t` vs `int64_t` vs `BigInt`), avoiding unnecessary BigInt allocations. |
||||
|
||||
3. **Overflow tracking:** Precisely determine when conversion from int64 to float/BigInt is needed, inserting conversion code only when overflow is actually possible. |
||||
|
||||
4. **Per-opcode type update table:** The design of `_zend_update_type_info()` maps directly onto AOT's Rule system — each opcode corresponds to a Rule responsible for outputting the result type of that operation. |
||||
|
||||
--- |
||||
|
||||
## 4. SCCP (Sparse Conditional Constant Propagation) |
||||
|
||||
### Core Design |
||||
|
||||
SCCP performs both **constant propagation** and **conditional constant folding** simultaneously, and can also eliminate unreachable code (no separate dead code elimination pass needed). |
||||
|
||||
Implemented in `sccp.c`, based on `scdf.h` (the SCDF framework). |
||||
|
||||
### Value Lattice |
||||
|
||||
``` |
||||
TOP (undefined) |
||||
/ | \ |
||||
C1 C2 C3 (constant values) |
||||
\ | / |
||||
BOT (overdefined = not constant) |
||||
``` |
||||
|
||||
- TOP: the value of this variable is not yet known (optimistic assumption) |
||||
- BOT: this variable is known not to be a constant |
||||
- Constant value: the exact value is known |
||||
|
||||
### Key Algorithm Points (from the comments at sccp.c:30-74) |
||||
|
||||
**The `meet` operation (merging of φ functions):** |
||||
- BOT + any = BOT |
||||
- TOP + any = any |
||||
- C_i + C_i = C_i (two identical constants) |
||||
- C_i + C_j = BOT (two different constants) |
||||
|
||||
**Instruction evaluation:** |
||||
- Any operand is BOT → result is BOT (exception: op1 of ASSIGN) |
||||
- Instructions that can never be evaluated → BOT |
||||
- Any operand is TOP → result is TOP |
||||
- All operands are known constants → attempt compile-time evaluation → return constant value on success, BOT on failure |
||||
|
||||
**Branch feasibility determination:** |
||||
- Branch on BOT → all successors are feasible |
||||
- Branch on TOP → no successor is infeasible (wait for more information) |
||||
- Branch on a known constant → only the branch that satisfies the condition is feasible |
||||
|
||||
### The SCDF Framework (`scdf.h`) |
||||
|
||||
SCCP is built on top of the SCDF (Sparse Conditional Data Flow) framework, a generic sparse conditional data-flow analysis engine: |
||||
|
||||
```c |
||||
typedef struct _scdf_ctx { |
||||
zend_op_array *op_array; |
||||
zend_ssa *ssa; |
||||
zend_bitset instr_worklist; // Instructions to process |
||||
zend_bitset phi_var_worklist; // Phi/SSA variables to process |
||||
zend_bitset block_worklist; // Blocks to process |
||||
zend_bitset executable_blocks; // Executable blocks |
||||
zend_bitset feasible_edges; // Feasible edges |
||||
|
||||
struct { |
||||
void (*visit_instr)(...); // Process an instruction |
||||
void (*visit_phi)(...); // Process a φ function |
||||
void (*mark_feasible_successors)(...); // Mark feasible successors |
||||
} handlers; |
||||
} scdf_ctx; |
||||
``` |
||||
|
||||
**Usage pattern:** SCCP implements `visit_instr` (constant evaluation), `visit_phi` (the meet operation), and `mark_feasible_successors` (branch feasibility). Type inference also uses a similar worklist propagation algorithm. |
||||
|
||||
**Generic worklist mechanism:** |
||||
```c |
||||
// When a variable's value changes, add all its uses to the worklist |
||||
static inline void scdf_add_to_worklist(scdf_ctx *scdf, int var_num) { |
||||
const zend_ssa_var *var = &ssa->vars[var_num]; |
||||
int use; |
||||
FOREACH_USE(var, use) { |
||||
zend_bitset_incl(scdf->instr_worklist, use); // Mark instructions using this variable |
||||
} |
||||
FOREACH_PHI_USE(var, phi) { |
||||
zend_bitset_incl(scdf->phi_var_worklist, phi->ssa_var); |
||||
} |
||||
} |
||||
``` |
||||
|
||||
### AOT Takeaways |
||||
|
||||
1. **The SCDF framework is the most directly reusable:** about 300 lines of C code providing a generic worklist-driven conditional data-flow engine. AOT can port it as a PHP class, reusing it across multiple optimization passes such as SCCP, type inference, and escape analysis. |
||||
|
||||
2. **The TOP/BOT lattice model:** The AOT compiler can use the same lattice structure when analyzing types: |
||||
- TOP = unknown type (early in analysis) |
||||
- BOT = contradictory type (an inconsistency was found) |
||||
- Concrete value (a constant or an exact type) |
||||
|
||||
3. **Conditional branch feasibility:** SCCP's branch feasibility determination can directly help AOT eliminate unreachable branches at compile time, generating simpler C++ code. |
||||
|
||||
--- |
||||
|
||||
## 5. DCE (Dead Code Elimination) |
||||
|
||||
### Algorithm (`dce.c`) |
||||
|
||||
PHP's DCE uses an optimistic strategy: |
||||
|
||||
``` |
||||
1. Assume all instructions and φ functions are dead |
||||
2. Mark all instructions with obvious side effects as live (side-effect instructions) |
||||
3. Starting from live instructions, mark the defining instructions of their operands as live (reverse propagation along use-def chains) |
||||
4. Repeat until the worklist is empty |
||||
5. Delete all instructions still marked as dead |
||||
``` |
||||
|
||||
**The key `may_have_side_effects()` check (`dce.c:74-100`):** |
||||
|
||||
Zend opcodes are divided into three kinds: |
||||
- Never have side effects (such as ADD, CONCAT, BOOL_NOT): can be eliminated by DCE |
||||
- May produce a notice but have no essential side effect (such as DIV_BY_ZERO triggering a warning): configurable whether to eliminate |
||||
- Always have side effects (such as ECHO, THROW, ASSIGN_OBJ): must be preserved |
||||
|
||||
**Special capability:** It can eliminate "redundant modifications to non-escaping arrays/objects" and "useless array/object allocations". If an array is only built, modified, and used locally, the intermediate ASSIGN_DIM steps may be eliminated. |
||||
|
||||
### AOT Takeaways |
||||
|
||||
The AOT compiler's DCE can be more aggressive (because types are known at compile time): |
||||
|
||||
1. **Side-effect classification matrix:** Build a side-effect table for AOT's expression/statement types, precisely marking which operations must be preserved |
||||
2. **Escape-aware DCE:** Combined with escape analysis, eliminate operations on non-escaping objects — this is one of AOT's biggest optimization opportunities |
||||
3. **Control-dependence based DCE:** PHP explicitly states that its current DCE does not consider control dependence (comments at `dce.c:35-39`); AOT can perform more precise control-dependence DCE |
||||
|
||||
--- |
||||
|
||||
## 6. Escape Analysis |
||||
|
||||
### Algorithm (`escape_analysis.c`) |
||||
|
||||
Based on the classic escape analysis algorithm of Kotzmann & Mossenbock (PPPJ'05). |
||||
|
||||
**Core steps:** |
||||
|
||||
1. **Build equivalence escape sets (`zend_build_equi_escape_sets`):** Uses the Union-Find algorithm. If two SSA variables are related through a φ function or ASSIGN (same value), they belong to the same equivalence class. |
||||
|
||||
2. **Escape state propagation:** Each equivalence class has four states: |
||||
``` |
||||
ESCAPE_STATE_UNKNOWN → initial state (zero-initialized C memory) |
||||
ESCAPE_STATE_NO_ESCAPE → definitely does not escape (final goal) |
||||
ESCAPE_STATE_FUNCTION_ESCAPE → escapes to the called function (passed as argument) |
||||
ESCAPE_STATE_GLOBAL_ESCAPE → global escape (returned, assigned to a global variable, throws an exception, etc.) |
||||
``` |
||||
|
||||
3. **Monotonic state convergence:** States can only go from UNKNOWN → NO_ESCAPE/FUNCTION_ESCAPE/GLOBAL_ESCAPE, never reverse. |
||||
|
||||
4. **Apply escape information:** |
||||
- Non-escaping arrays can be allocated on the stack (no heap allocation needed) |
||||
- Non-escaping objects can avoid reference counting operations |
||||
- Non-escaping variables do not need separation (ZEND_SEPARATE) |
||||
|
||||
**Predecessor/successor edges:** Supports symbolic type aliases (SYMTABLE_ALIAS) and HTTP response header aliases (HTTP_RESPONSE_HEADER_ALIAS). |
||||
|
||||
### AOT Takeaways |
||||
|
||||
Escape analysis may be the most valuable for the AOT compiler: |
||||
|
||||
1. **Box allocation elimination:** AOT uses `Box<T>` to represent object references. Escape analysis can confirm which Boxes do not need heap allocation and can be created directly on the stack. |
||||
|
||||
2. **Reference count elimination:** Non-escaping objects can skip `php::Object::Ref()` / `php::Object::Unref()` operations. |
||||
|
||||
3. **Array stack allocation:** After escape analysis, local arrays can use a stack-based `zend_array`. |
||||
|
||||
4. **The 4-state model is very simple and effective**, and AOT can map it directly: |
||||
- ESCAPE_STATE_NO_ESCAPE → stack allocation |
||||
- ESCAPE_STATE_FUNCTION_ESCAPE → decided by the caller |
||||
- ESCAPE_STATE_GLOBAL_ESCAPE → heap allocation |
||||
|
||||
--- |
||||
|
||||
## 7. Call Graph |
||||
|
||||
### Design (`zend_call_graph.h`) |
||||
|
||||
PHP's call graph tracks bidirectional relationships, both caller → callee and callee → caller: |
||||
|
||||
```c |
||||
struct _zend_call_info { |
||||
zend_op_array *caller_op_array; // Caller |
||||
zend_op *caller_init_opline; // INIT_FCALL instruction |
||||
zend_op *caller_call_opline; // DO_FCALL instruction |
||||
zend_function *callee_func; // Called function |
||||
zend_call_info *next_caller; // Linked list: the callee's next caller |
||||
zend_call_info *next_callee; // Linked list: the caller's next callee |
||||
bool recursive; // Recursive call |
||||
bool send_unpack; // Uses SEND_UNPACK |
||||
bool named_args; // Named arguments |
||||
bool is_prototype; // May be a method overridden by a subclass |
||||
bool is_frameless; // frameless function |
||||
int num_args; |
||||
zend_send_arg_info arg_info[1]; |
||||
}; |
||||
|
||||
struct _zend_func_info { |
||||
zend_ssa ssa; // The function's own SSA |
||||
zend_call_info *caller_info; // Who called this function |
||||
zend_call_info *callee_info; // Whom this function called |
||||
zend_call_info **call_map; // Quick index from opnum to call_info |
||||
zend_ssa_var_info return_info; // Inferred return type |
||||
}; |
||||
``` |
||||
|
||||
**Key features:** |
||||
|
||||
1. **Bidirectional graph:** `caller_info` and `callee_info` are separate linked lists, supporting traversal upward (from callee to find callers) and downward (from caller to find callees) |
||||
2. **call_map:** An array indexing opnum → call_info, providing O(1) lookup of the call information for a given opcode position |
||||
3. **Return type propagation:** The callee's return_info can be propagated upward to the caller's return_info |
||||
4. **Argument type propagation:** The caller's actual argument types can be propagated downward to the callee's parameter types (for more precise function-body optimization) |
||||
|
||||
### Advanced Cross-Function Optimization (`zend_optimize_script:1626-1728`) |
||||
|
||||
``` |
||||
1. build_call_graph → build the bidirectional call graph |
||||
2. zend_optimize (per-func) → perform independent local optimization per function |
||||
3. analyze_call_graph → infer function information (recursion flags, indirect variable access, func_get_args, etc.) |
||||
4. build_call_map → build the opnum→call index for each function |
||||
5. dfa_analyze_op_array → build SSA + type inference (per-func) |
||||
6. dfa_optimize_op_array → perform SCCP + DCE + block pass based on SSA |
||||
``` |
||||
|
||||
### AOT Takeaways |
||||
|
||||
The AOT compiler's first two steps (prepare + convert) naturally build a complete symbol dependency graph. On top of this, it can add: |
||||
|
||||
1. **call_map index:** Quickly look up the callee's metadata from each call site (parameter types, return type, whether it is an inlining candidate) |
||||
2. **Bidirectional return type propagation:** AOT's return type inference is currently top-down; the call graph allows feeding the callee's known return type back to the caller |
||||
3. **Recursion flags:** `ZEND_FUNC_RECURSIVE_DIRECTLY` / `ZEND_FUNC_RECURSIVE_INDIRECTLY` are critical for inlining strategy decisions |
||||
|
||||
--- |
||||
|
||||
## 8. JIT IR Framework |
||||
|
||||
### Design |
||||
|
||||
The IR (Intermediate Representation) used by PHP JIT is a generic SSA-derived low-level intermediate representation, located in `ext/opcache/jit/ir/`. |
||||
|
||||
**Three stages of IR:** |
||||
|
||||
| Stage | Files | Purpose | |
||||
|------|------|------| |
||||
| IR builder | `ir_builder.h`, `zend_jit_ir.c` | Build IR instructions from Zend bytecode | |
||||
| IR optimizer | `ir_cfg.c`, `ir_fold.h`, `ir_gcm.c` | CFG optimization, constant folding, global code motion (GCM) | |
||||
| IR emitter | `ir_emit.c`, `ir_emit_x86.h` | Emit x86/ARM64 machine code from IR | |
||||
|
||||
**Example IR instructions: IR_ADD, IR_MUL, IR_LOAD, IR_STORE, IR_CALL, IR_GUARD, etc.** |
||||
|
||||
**JIT optimization levels (`zend_jit.h:32-37`):** |
||||
```c |
||||
#define ZEND_JIT_LEVEL_NONE 0 // JIT not enabled |
||||
#define ZEND_JIT_LEVEL_MINIMAL 1 // Minimal JIT (subroutine threading) |
||||
#define ZEND_JIT_LEVEL_INLINE 2 // Selective inline threading |
||||
#define ZEND_JIT_LEVEL_OPT_FUNC 3 // Optimize a single function based on type inference |
||||
#define ZEND_JIT_LEVEL_OPT_FUNCS 4 // Optimize based on the call tree |
||||
#define ZEND_JIT_LEVEL_OPT_SCRIPT 5 // Interprocedural analysis |
||||
``` |
||||
|
||||
**JIT trigger modes (`zend_jit.h:39-44`):** |
||||
```c |
||||
#define ZEND_JIT_ON_SCRIPT_LOAD 0 // Compile immediately when all functions are loaded |
||||
#define ZEND_JIT_ON_FIRST_EXEC 1 // Compile on first execution |
||||
#define ZEND_JIT_ON_PROF_REQUEST 2 // Compile the hottest functions based on profile data |
||||
#define ZEND_JIT_ON_HOT_COUNTERS 3 // Compile after N calls/loop iterations |
||||
#define ZEND_JIT_ON_HOT_TRACE 5 // Use tracing JIT after N calls |
||||
``` |
||||
|
||||
### AOT Takeaways |
||||
|
||||
1. **IR as an intermediate carrier for AST→C++:** AOT currently generates C++ code directly from the AST. Introducing an IR layer can: |
||||
- Perform optimization at the IR level (fold, GCM, register allocation simulation) |
||||
- Decouple the front end (PHP AST) from the back end (C++ codegen) |
||||
|
||||
2. **The constant folding table of `ir_fold.h`:** IR contains an auto-generated folding rule table (`gen_ir_fold_hash`) defining hundreds of algebraic simplification rules. AOT can adopt a similar "rule table"-driven constant folding approach. |
||||
|
||||
3. **JIT's profiling mechanism:** `hot_loop` / `hot_func` counters — AOT can embed profile data into the generated binary for PGO (Profile-Guided Optimization). |
||||
|
||||
--- |
||||
|
||||
## 9. OPcache Persistence & File Cache |
||||
|
||||
### Design |
||||
|
||||
OPcache is not just a cache — it stores the **optimized** bytecode in the cache. |
||||
|
||||
``` |
||||
Original PHP source code |
||||
→ compiled to zend_op_array (original bytecode) |
||||
→ through all Zend Optimizer passes (SSA + type inference + SCCP + DCE + ...) |
||||
→ keep only the optimized zend_op_array (discard temporary IR such as SSA) |
||||
→ zend_persist() serializes to shared memory / file cache |
||||
``` |
||||
|
||||
**`zend_persist_calc` + `zend_persist`**: two-phase serialization — |
||||
1. `_calc` computes the required shared memory size |
||||
2. `_persist` performs the actual serialization (all pointers adjusted to absolute offsets) |
||||
|
||||
**File cache (`zend_file_cache.c`):** Writes the persisted script to a file, allowing reuse across process restarts. |
||||
|
||||
### AOT Takeaways |
||||
|
||||
The current AOT compiler compiles from PHP source code every time. It can borrow the OPcache philosophy: |
||||
|
||||
1. **Cache the optimized AST/type information:** After the `convert()` stage, serialize the typed AST and load it directly on the next compilation |
||||
2. **Incremental compilation:** Recompile only changed files and their dependencies |
||||
3. **Two-phase serialization (`_calc` + `_persist`):** Compute the size first, then allocate memory/write, avoiding realloc fragmentation |
||||
|
||||
--- |
||||
|
||||
## 10. Other Notable Designs |
||||
|
||||
### zend_bitset |
||||
|
||||
PHP uses its own bitset implementation for efficient set operations. The optimizer frequently uses bitsets to represent worklists, live sets, and def/use sets. |
||||
|
||||
### zend_worklist.h |
||||
|
||||
Generic worklist iteration macros; SCCP and type inference use the same worklist mechanism. Designed as macros for inlining performance. |
||||
|
||||
### zend_arena |
||||
|
||||
An arena memory allocator used for fast allocation and bulk release of all optimizer data structures. One arena is bound to one `zend_optimizer_ctx`, and all passes share the same arena. |
||||
|
||||
### Inter-Pass Data Management |
||||
|
||||
Temporary IR such as SSA is destroyed immediately after a pass completes (via arena free), keeping only the optimized results in the final `zend_op_array`. This guarantees memory efficiency. |
||||
|
||||
--- |
||||
|
||||
## Recommended Adoption Order (for the AOT compiler) |
||||
|
||||
``` |
||||
Phase 1: Pass Pipeline |
||||
└── Define the AotPass enum + Pipeline runner, pluggable pass architecture |
||||
|
||||
Phase 2: Bitmask type system |
||||
└── Borrow Zend's type mask design, replacing the current string type constants |
||||
└── Map directly to C++ uint32_t constants |
||||
|
||||
Phase 3: Type inference rules |
||||
└── Each AST node/opcode corresponds to an update_type_info |
||||
└── Implement in combination with the Rule system (#4 design) |
||||
|
||||
Phase 4: SCCP + DCE |
||||
└── Constant propagation + dead code elimination based on the SCDF framework |
||||
└── Can eliminate redundant expressions before C++ code generation |
||||
|
||||
Phase 5: Escape analysis |
||||
└── Union-Find equivalence escape sets + 4-state propagation |
||||
└── Used for Box allocation elimination and reference count elimination |
||||
|
||||
Phase 6: Call graph cross-function optimization |
||||
└── Add call_map + type propagation on top of the existing symbol dependency graph |
||||
``` |
||||
|
||||
Each layer can be implemented independently and immediately bring benefits to existing code generation. |
||||
@ -0,0 +1,559 @@ |
||||
# PHPStan Design Analysis: Patterns and Modules Portable to the AOT Compiler |
||||
|
||||
This document analyzes the architecture of the PHPStan project (`projects/phpstan-src/`) and identifies design patterns and modules that can be introduced into the AOT compiler. |
||||
|
||||
--- |
||||
|
||||
## Overview: Migratable Designs Across Four Priorities |
||||
|
||||
| Priority | Design / Module | Implementation Size | Prerequisites | Benefit | |
||||
|--------|----------|---------|---------|------| |
||||
| P1 | Type object hierarchy (#1) | ~1000 lines | None | Qualitative leap in type inference precision | |
||||
| P1 | TrinaryLogic (#2) | ~80 lines | None | Type queries no longer return wrong answers | |
||||
| P1 | TypeCombinator (#3) | ~300-400 lines | #1 | Eliminate scattered type string concatenation | |
||||
| P2 | Rule system (#4) | Interface + Registry ~80 lines | Mostly independent | Architectural modularity, easy to test and extend | |
||||
| P2 | Collector two-phase analysis (#5) | ~200 lines | #4 | Cross-file global optimization | |
||||
| P2 | Extension registration mechanism (#6) | ~100 lines | #4 | Framework-level plugin system | |
||||
| P3 | TypeSpecifier / type narrowing (#7) | ~200 lines | #1, #2 | if/else branch type refinement | |
||||
| P3 | Immutable Scope (#8) | ~300 lines | #1, #2 | Expression-level type tracking | |
||||
| P4 | PHPDoc Pipeline (#9) | Reuse phpstan/phpdoc-parser | #1 | Leverage existing mature parser | |
||||
| P4 | NeverType (#10) | Contained in #1 | #1 | Dead code elimination, contradictory type detection | |
||||
|
||||
--- |
||||
|
||||
## 1. Object-Oriented Type System (Replacing String-Based Type Representation) |
||||
|
||||
### Current State |
||||
|
||||
The AOT compiler represents types with string constants: |
||||
|
||||
```php |
||||
const TYPE_INT = 'int'; |
||||
const TYPE_FLOAT = 'float'; |
||||
const TYPE_STR = 'string'; |
||||
// Compound types use string concatenation |
||||
// 'int|string', 'string|null' |
||||
``` |
||||
|
||||
This forces type operations (merging, comparison, querying) to manually parse and concatenate strings. |
||||
|
||||
### PHPStan's Approach |
||||
|
||||
PHPStan has a `Type` interface defining roughly 100 methods. Each type is a class: |
||||
|
||||
``` |
||||
Type (interface) |
||||
├── StringType |
||||
├── IntegerType |
||||
├── FloatType |
||||
├── BooleanType |
||||
├── NullType |
||||
├── MixedType (top type, with subtracted type support) |
||||
├── NeverType (bottom type, no possible values) |
||||
├── VoidType |
||||
├── ArrayType |
||||
├── ObjectType |
||||
├── CallableType |
||||
├── UnionType (A|B|C, holding a flat list<Type>) |
||||
├── IntersectionType (A&B&C) |
||||
├── ConstantStringType, ConstantIntegerType, ... |
||||
├── IntegerRangeType |
||||
└── Accessory*Type (for refinement inside IntersectionType) |
||||
``` |
||||
|
||||
**Core design principle: never use `instanceof` to determine type identity.** You must use `is*()` methods: |
||||
|
||||
```php |
||||
// Wrong — misses UnionType and IntersectionType |
||||
$type instanceof StringType |
||||
|
||||
// Correct — UnionType delegates to inner types and combines the results |
||||
$type->isString()->yes() |
||||
``` |
||||
|
||||
**Key sub-interfaces:** |
||||
|
||||
- `CompoundType`: marks types that need two-way type comparison (UnionType, IntersectionType). Adds `isAcceptedBy()`, `isSubTypeOf()` methods, implementing a double-dispatch protocol. |
||||
- `SubtractableType`: supports set difference operations (e.g. `mixed~null`). |
||||
|
||||
### The isSuperTypeOf / accepts Double-Dispatch Protocol |
||||
|
||||
This is the most central design pattern of PHPStan's type system: |
||||
|
||||
``` |
||||
Simple type (StringType)::isSuperTypeOf(Type $otherType): |
||||
if $otherType is a CompoundType: |
||||
return $otherType->isSubTypeOf($this) // reverse delegation |
||||
// own logic ... |
||||
return No |
||||
|
||||
Compound type (UnionType)::isSubTypeOf(Type $otherType): |
||||
foreach innerType in $this->types: |
||||
results[] = $otherType->isSuperTypeOf(innerType) |
||||
return extremeIdentity(results) // ALL must be subtype |
||||
``` |
||||
|
||||
**Key point:** simple types do not need to understand compound type semantics. Adding a new compound type does not require modifying any simple type's comparison logic. |
||||
|
||||
### AOT Adoption Recommendation |
||||
|
||||
The AOT compiler does not need PHPStan's full complexity. The first version only needs these concrete types: |
||||
|
||||
``` |
||||
IntegerType, FloatType, StringType, BoolType, |
||||
NullType, MixedType, NeverType, ArrayType, ObjectType, UnionType |
||||
``` |
||||
|
||||
Not needed: IntersectionType, AccessoryType, TemplateType, Constant*Type, IntegerRangeType, EnumType. |
||||
|
||||
--- |
||||
|
||||
## 2. TrinaryLogic (Three-Valued Logic) |
||||
|
||||
### Current State |
||||
|
||||
The AOT compiler uses boolean to determine type properties. But the `mixed` type means "could be string, could be int", and boolean cannot express this uncertainty. |
||||
|
||||
### PHPStan's Approach |
||||
|
||||
```php |
||||
class TrinaryLogic { |
||||
public function yes(): bool; |
||||
public function no(): bool; |
||||
public function maybe(): bool; |
||||
|
||||
public static function createYes(): self; |
||||
public static function createNo(): self; |
||||
public static function createMaybe(): self; |
||||
|
||||
public function and(self ...$others): self; |
||||
public function or(self ...$others): self; |
||||
public function extremeIdentity(self ...$others): self; // ALL yes → yes; ALL no → no |
||||
public function maxMin(self ...$others): self; // ANY yes → yes; ALL no → no |
||||
} |
||||
``` |
||||
|
||||
Usage example: |
||||
|
||||
```php |
||||
// MixedType::isString() → maybe (mixed may be string) |
||||
// IntegerType::isString() → no |
||||
// UnionType(int|string)::isString() → maybe |
||||
|
||||
$type->isString()->yes() // definitely a string |
||||
$type->isString()->no() // definitely not a string |
||||
$type->isString()->maybe() // uncertain |
||||
``` |
||||
|
||||
### AOT Adoption Recommendation |
||||
|
||||
Port directly, about 80 lines of code, no external dependencies. Replace boolean with TrinaryLogic for all type query methods. |
||||
|
||||
--- |
||||
|
||||
## 3. TypeCombinator — The Type Normalization Engine |
||||
|
||||
### Problem to Solve |
||||
|
||||
Forbid directly calling `new UnionType(...)`. All union/intersect/remove operations must go through TypeCombinator to ensure the type representation is always normalized. |
||||
|
||||
### Three Core Operations |
||||
|
||||
#### `union(Type ...$types): Type` |
||||
|
||||
**Algorithm flow:** |
||||
|
||||
1. **Fast path**: 0 arguments → `NeverType`; 1 argument → return directly; 2 arguments check `never`/`mixed`/identical objects |
||||
2. **Flattening**: `union(A, union(B, C), D)` → expands to `[A, B, C, D]` |
||||
3. **Filter NeverType**: `union(int, never, string)` → `[int, string]` |
||||
4. **Category extraction**: divide types into scalar/array/enum/integerRange/generic five categories, process in batches |
||||
5. **Scalar resolution**: `ConstantIntegerType(3) | IntegerType` → `IntegerType`; `true | false` → `BooleanType` |
||||
6. **Pairwise comparison**: |
||||
- Adjacent `IntegerRangeType` intervals merge: `int<0,5> | int<3,10>` → `int<0,10>` |
||||
- Subtype/supertype elimination: `Foo extends Bar` ⇒ `Foo | Bar` = `Bar` |
||||
- `int[] | string[]` → `(int|string)[]` |
||||
7. **Wrap-up**: 0 → NeverType; 1 → return directly; otherwise `new UnionType(array_values($types), true)` |
||||
|
||||
#### `intersect(Type ...$types): Type` |
||||
|
||||
**Algorithm flow:** |
||||
|
||||
1. If there is a `NeverType` → return never directly |
||||
2. **Distributive expansion**: `A & (B|C)` → `(A&B) | (A&C)`, then recurse on each term |
||||
3. **Flattening**: `A & (B & C)` → expands to `[A, B, C]` |
||||
4. **Pairwise two-way comparison**: |
||||
- `IntegerType & ConstantIntegerType(5)` → `ConstantIntegerType(5)` (Child & Parent = Child) |
||||
- `int & string` → `NeverType` (contradiction) |
||||
- SubtractableType difference handoff |
||||
5. **Contradiction detection**: `isSuperTypeOf` returns `no` → `NeverType` |
||||
|
||||
#### `remove(Type $fromType, Type $typeToRemove): Type` |
||||
|
||||
``` |
||||
remove(int|string, string) = int |
||||
remove(int|string|null, null) = int|string |
||||
remove(int, string) = int // what to remove is not present |
||||
remove(string, string) = never // fully removed |
||||
remove(mixed, Foo) = mixed~Foo // difference type |
||||
``` |
||||
|
||||
### AOT Adoption Recommendation |
||||
|
||||
A simplified TypeCombinator (~300-400 lines) covering union/intersect/remove for the basic types AOT needs. PHPStan's complex array shape handling, accessory type propagation, IntegerRange merging, etc. are not needed. |
||||
|
||||
### Benefits |
||||
|
||||
| Scenario | Current | After TypeCombinator | |
||||
|------|------|-------------------| |
||||
| Variable assignment of `int\|int` | String `'int\|int'` | Automatically simplified to `IntegerType` | |
||||
| Return value of `mixed\|string` | Does not know how to simplify | Automatically simplified to `MixedType` | |
||||
| Merging two branch types | Manual concatenation | `union()` automatically dedupes and removes subtypes | |
||||
| `int & string` intersection | Cannot detect | Returns `NeverType` (compile error) | |
||||
|
||||
--- |
||||
|
||||
## 4. Rule-Based Analysis Pass System |
||||
|
||||
### Core Interface |
||||
|
||||
```php |
||||
/** |
||||
* @template TNodeType of Node |
||||
*/ |
||||
interface Rule |
||||
{ |
||||
/** @return class-string<TNodeType> */ |
||||
public function getNodeType(): string; |
||||
|
||||
/** @param TNodeType $node */ |
||||
public function processNode(Node $node, Scope $scope): array; |
||||
} |
||||
``` |
||||
|
||||
Each Rule declares which AST node type it cares about and what to do when that node is found. |
||||
|
||||
### Registration Mechanism |
||||
|
||||
Declare level via PHP 8 Attribute: |
||||
|
||||
```php |
||||
#[RegisteredRule(level: 0)] |
||||
final class CallMethodsRule implements Rule |
||||
{ |
||||
public function getNodeType(): string { return MethodCall::class; } |
||||
public function processNode(Node $node, Scope $scope): array { ... } |
||||
} |
||||
``` |
||||
|
||||
### Registry Implementation |
||||
|
||||
`LazyRegistry` collects all services tagged `phpstan.rules.rule` from the DI container and indexes them by the `getNodeType()` return value. |
||||
|
||||
**Key design:** `getRules($nodeType)` matches not only the exact class name, but also all parent classes and interfaces: |
||||
|
||||
```php |
||||
public function getRules(string $nodeType): array |
||||
{ |
||||
// $nodeType = MethodCall::class |
||||
// parentNodeTypes = [MethodCall, Expr, NodeAbstract, Node, ...] |
||||
// match all Rules registered on these parent classes/interfaces |
||||
$parentNodeTypes = [$nodeType] + class_parents($nodeType) + class_implements($nodeType); |
||||
// ... |
||||
} |
||||
``` |
||||
|
||||
This means a rule registered for `Node\Expr` matches **all** expression types. |
||||
|
||||
### Runtime Dispatch |
||||
|
||||
At every node during AST traversal: |
||||
|
||||
```php |
||||
$nodeType = get_class($node); |
||||
foreach ($this->ruleRegistry->getRules($nodeType) as $rule) { |
||||
$ruleErrors = $rule->processNode($node, $scope); |
||||
// transform and collect errors ... |
||||
} |
||||
``` |
||||
|
||||
Extremely concise — no switch, no if-else chains. |
||||
|
||||
### Collector Two-Phase Analysis |
||||
|
||||
The Collector interface is almost identical to Rule, but returns collected data (instead of errors): |
||||
|
||||
```php |
||||
interface Collector |
||||
{ |
||||
public function getNodeType(): string; |
||||
/** @return TValue|null */ |
||||
public function processNode(Node $node, Scope $scope); |
||||
} |
||||
``` |
||||
|
||||
**Phase 1 (per-file):** as each file is analyzed, the collector gathers data. |
||||
|
||||
**Phase 2 (global):** after all files are analyzed, create a `CollectedDataNode` wrapping all data and run the rules registered on it: |
||||
|
||||
```php |
||||
$node = new CollectedDataNode($analyserResult->getCollectedData(), $onlyFiles); |
||||
foreach ($this->ruleRegistry->getRules(CollectedDataNode::class) as $rule) { |
||||
$ruleErrors = $rule->processNode($node, $scope); |
||||
} |
||||
``` |
||||
|
||||
`CollectedDataNode::get(string $collectorType): array<string, list<TValue>>` indexes collected data by file path. |
||||
|
||||
### Concrete Application in the AOT Compiler |
||||
|
||||
| Rule | Trigger Node | Purpose | |
||||
|------|----------|------| |
||||
| `BinaryOpCodegenRule` | `Expr\BinaryOp` | Generate C++ operation code | |
||||
| `MethodCallCodegenRule` | `Expr\MethodCall` | Method call code generation, virtual/direct call determination | |
||||
| `TypeCheckInsertRule` | `Param` / `Return_` | Insert runtime type checks at function entry/exit | |
||||
| `DeadCodeEliminateRule` | `CollectedDataNode` | Cross-file analysis, remove uncalled functions | |
||||
| `DevirtualizeRule` | `CollectedDataNode` | Single-implementation virtual methods → direct calls | |
||||
| `InlineDecisionRule` | `CollectedDataNode` | Decide inlining based on call frequency and function size | |
||||
| `ConstantFoldRule` | `Expr\BinaryOp` | Compile-time constant folding | |
||||
| `BoxOptimizationRule` | `Expr\Assign` | Box escape analysis for std containers | |
||||
|
||||
**O0/O1/O2 tiers:** |
||||
|
||||
```php |
||||
#[RegisteredRule(level: 0)] // basic code generation, always required |
||||
class BinaryOpCodegenRule implements Rule { ... } |
||||
|
||||
#[RegisteredRule(level: 1)] // O1 optimization |
||||
class ConstantFoldRule implements Rule { ... } |
||||
|
||||
#[RegisteredRule(level: 2)] // O2 aggressive optimization |
||||
class InlineDecisionRule implements Rule { ... } |
||||
``` |
||||
|
||||
### AOT Adoption Recommendation |
||||
|
||||
Progressive migration, no need to rewrite the entire compiler at once: |
||||
|
||||
1. Define the `Rule` interface and `Registry` (about 80 lines of code) |
||||
2. Extract one standalone function of `parseStmts()` as the first Rule |
||||
3. Migrate the remaining switch branches step by step |
||||
4. Introduce `Collector` + `CollectedDataNode` for cross-file optimization |
||||
|
||||
The Rule system can coexist with the existing switch dispatch — let Rules first handle the nodes they can, and fall back to the original logic for the rest. |
||||
|
||||
--- |
||||
|
||||
## 5. Extension Registration Mechanism |
||||
|
||||
### PHPStan's Approach |
||||
|
||||
PHPStan has dozens of extension interfaces, registered through the DI container's service tag mechanism: |
||||
|
||||
``` |
||||
DynamicMethodReturnTypeExtension → tag: phpstan.broker.dynamicMethodReturnTypeExtension |
||||
FunctionTypeSpecifyingExtension → tag: phpstan.typeSpecifier.functionTypeSpecifyingExtension |
||||
TypeNodeResolverExtension → tag: phpstan.phpdoc.typeNodeResolverExtension |
||||
MethodsClassReflectionExtension → tag: phpstan.broker.methodsClassReflectionExtension |
||||
PropertiesClassReflectionExtension → tag: phpstan.broker.propertiesClassReflectionExtension |
||||
... |
||||
``` |
||||
|
||||
Extensions are called before the core logic and get a chance to override the default behavior: |
||||
|
||||
```php |
||||
public function resolve(TypeNode $typeNode, NameScope $nameScope): ?Type |
||||
{ |
||||
foreach ($this->extensions as $extension) { |
||||
$type = $extension->resolve($typeNode, $nameScope); |
||||
if ($type !== null) { |
||||
return $type; // extension handled it, short-circuit core logic |
||||
} |
||||
} |
||||
// core logic ... |
||||
} |
||||
``` |
||||
|
||||
### AOT Application |
||||
|
||||
Framework-specific compilation optimizations can be provided through extension plugins instead of modifying the compiler core: |
||||
|
||||
```php |
||||
interface MethodCallOptimizationExtension |
||||
{ |
||||
/** Returns optimized C++ code, or null to indicate no handling */ |
||||
public function optimize(MethodCall $call, Scope $scope): ?string; |
||||
} |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 6. TypeSpecifier — The Type Narrowing Engine |
||||
|
||||
### Problem Solved |
||||
|
||||
When the AOT compiler encounters `if ($x instanceof Foo)`, inside the branch `$x`'s type should be narrowed to `Foo`. This enables generating more efficient C++ code (directly calling Foo's methods without going through the vtable). |
||||
|
||||
### PHPStan's Approach |
||||
|
||||
`TypeSpecifier` analyzes conditional expressions and decides how to narrow types in the truthy/falsy branches: |
||||
|
||||
| Condition | Truthy narrows to | Falsey narrows to | |
||||
|------|-------------|--------------| |
||||
| `$x instanceof Foo` | `Foo` | Remove `Foo` | |
||||
| `$x === null` | `NullType` | Remove `NullType` | |
||||
| `is_array($x)` | `ArrayType` | Remove `ArrayType` | |
||||
| `$x` (truthy) | Remove falsey types | Keep only falsey types | |
||||
| `$x > 0` | `int<1, max>` | `int<min, 0>` | |
||||
|
||||
### Type Narrowing Pipeline |
||||
|
||||
``` |
||||
Conditional expression |
||||
→ TypeSpecifier::specifyTypesInCondition(scope, expr, context) |
||||
→ SpecifiedTypes { sureTypes[], sureNotTypes[] } |
||||
→ MutatingScope::filterBySpecifiedTypes(types) |
||||
→ new scope (types already narrowed) |
||||
``` |
||||
|
||||
### AOT Application |
||||
|
||||
A simplified TypeSpecifier (~200 lines) handling: |
||||
- `instanceof` narrowing |
||||
- `=== null` / `!== null` narrowing |
||||
- `is_array()`, `is_string()`, `is_int()` and other function narrowing |
||||
- `BooleanAnd`/`BooleanOr` chained narrowing |
||||
|
||||
--- |
||||
|
||||
## 7. Immutable Scope |
||||
|
||||
### Current State |
||||
|
||||
AOT's `FunctionContext` uses mutable public arrays and tracks types only by variable name: |
||||
|
||||
```php |
||||
class FunctionContext { |
||||
public array $localVars = []; // only variable name → type |
||||
public int $scopeLevel = 0; // simple nesting counter |
||||
public array $scopeLayouts = []; // ScopeContext is an empty class |
||||
public bool $inLoop = false; |
||||
public bool $inClosure = false; |
||||
} |
||||
``` |
||||
|
||||
### PHPStan's Approach |
||||
|
||||
`MutatingScope` is an **immutable** persistent data structure. Every change returns a new instance: |
||||
|
||||
``` |
||||
scope.assignVariable('x', stringType) → new scope |
||||
scope.filterByTruthyValue(instanceofExpr) → new scope (narrowed types) |
||||
scope.filterByFalseyValue(instanceofExpr) → new scope (removed types) |
||||
scope.mergeWith(elseScope) → new scope (intersection) |
||||
``` |
||||
|
||||
Tracks types by **expression string**, not just by variable name: |
||||
|
||||
``` |
||||
'$a' → ExpressionTypeHolder($a, IntegerType, Yes) |
||||
'$a[0]' → ExpressionTypeHolder($a[0], StringType, Yes) |
||||
'$a->prop' → ExpressionTypeHolder($a->prop, FooType, Yes) |
||||
'strlen($a)' → ExpressionTypeHolder(strlen($a), IntegerType, Yes) |
||||
``` |
||||
|
||||
This means assigning to `$a` invalidates the type caches for `$a[0]` and `$a->prop`. |
||||
|
||||
### AOT Adoption Recommendation |
||||
|
||||
A simplified version (~300 lines) with core capabilities: |
||||
|
||||
- Immutable scope supporting snapshots and merging |
||||
- Type tracking by expression key (at least variables and array elements) |
||||
- `TrinaryLogic` certainty tracking (the maybe state after branch merging) |
||||
|
||||
--- |
||||
|
||||
## 8. PHPDoc Pipeline |
||||
|
||||
### PHPStan's Approach |
||||
|
||||
A three-stage pipeline: |
||||
|
||||
``` |
||||
PHPDoc comment string |
||||
→ Lexer + Parser (phpstan/phpdoc-parser) |
||||
→ PhpDocNode (raw AST) |
||||
→ TypeNodeResolver::resolve(TypeNode, NameScope) |
||||
→ PHPStan Type object |
||||
``` |
||||
|
||||
`TypeNodeResolver` dispatches via a `switch` to 30+ identifier types: |
||||
|
||||
``` |
||||
'int' / 'integer' → IntegerType |
||||
'positive-int' → IntegerRangeType(1, null) |
||||
'non-empty-string' → IntersectionType[StringType, AccessoryNonEmptyStringType] |
||||
'class-string' → ClassStringType |
||||
'array' → ArrayType(MixedType, MixedType) |
||||
'list' → IntersectionType[ArrayType(int), AccessoryArrayListType] |
||||
'mixed' → MixedType(true) |
||||
'never' → NonAcceptingNeverType |
||||
... |
||||
``` |
||||
|
||||
### AOT Application |
||||
|
||||
AOT can reuse `phpstan/phpdoc-parser` to parse `@param`, `@return`, `@var` annotations, then map the type AST nodes to AOT's own Type system. `NameScope` (tracking namespace + use imports) is also a directly reusable concept. |
||||
|
||||
--- |
||||
|
||||
## 9. NeverType (Bottom Type) |
||||
|
||||
Represents a type that "cannot have any value", i.e. the empty set: |
||||
|
||||
``` |
||||
union(int, never) = int // never is the identity element of union |
||||
intersect(string, never) = never // never is the absorbing element of intersect |
||||
``` |
||||
|
||||
### AOT Application |
||||
|
||||
- Dead code elimination: an expression narrowed to NeverType → that path is unreachable |
||||
- Error propagation: contradictory type combinations produce NeverType |
||||
- `void` function returns: essentially NeverType in the return position |
||||
|
||||
--- |
||||
|
||||
## Recommended Adoption Order |
||||
|
||||
``` |
||||
Phase 1 (foundation): Type interface + concrete types + TypeCombinator + TrinaryLogic |
||||
└── Replace string-based type representation, expected to reduce scattered type string operations |
||||
|
||||
Phase 2 (modularization): Rule interface + Registry + Attribute registration |
||||
└── Break down the huge switch of parseStmts/parseExpr |
||||
|
||||
Phase 3 (analysis): TypeSpecifier + basic Immutable Scope |
||||
└── Enable if/else branch type narrowing |
||||
|
||||
Phase 4 (optimization): Collector + CollectedDataNode |
||||
└── Enable cross-file global optimization (devirtualization, dead code elimination, inline decisions) |
||||
|
||||
Phase 5 (ecosystem): Extension registration + PHPDoc Pipeline |
||||
└── Enable framework plugins and annotation-driven type information |
||||
``` |
||||
|
||||
Each layer builds on the previous one, and each layer can deliver value independently. |
||||
|
||||
--- |
||||
|
||||
## PHPStan Features That Should NOT Be Introduced |
||||
|
||||
| Feature | Reason | |
||||
|------|------| |
||||
| Generics / `@template T` | Extremely complex, AOT does not currently need it | |
||||
| IntersectionType + Accessory types | `non-empty-string` = `string & non-empty` is an over-engineered design | |
||||
| BetterReflection (static reflection) | AOT already loads files, no need to avoid runtime side effects | |
||||
| Full MutatingScope (5000+ lines) | The immutable scope pattern is worth borrowing, but 300 lines is enough | |
||||
| ConditionalExpressionHolder | Lazy narrowing of compound conditions; do basic TypeSpecifier first | |
||||
| Enum / ConstantArrayType shape | Niche, not needed initially | |
||||
@ -0,0 +1,986 @@ |
||||
# TypePHP and Python Language-Level Interop Design |
||||
|
||||
> Status: the core design is confirmed and is being implemented in phases according to `python/implementation-plan.md`. |
||||
> |
||||
> This document is the design specification for syntax, type semantics, runtime boundaries, and compatibility goals; details that are not yet confirmed continue to be maintained at the end of the document. |
||||
|
||||
## 1. Goals |
||||
|
||||
TypePHP should provide, at the language level, the ability to call Python packages from TypePHP: |
||||
|
||||
1. TypePHP imports Python modules, accesses module members, and calls Python functions and classes. |
||||
2. TypePHP operates on Python objects, including attributes, methods, subscripts, iteration, operators, and the call protocol. |
||||
3. TypePHP functions, closures, and objects can be passed as arguments to Python calls, and Python is allowed to synchronously call back into TypePHP within that dynamic call relationship. |
||||
4. The two VMs interoperate directly within the same process, without going through JSON, RPC, or subprocesses. |
||||
5. Python object identity and type information are preserved by default, avoiding unnecessary deep copies. |
||||
6. The syntax targets ordinary TypePHP/PHP developers; routine calls do not require understanding the CPython C API, the GIL, or reference counting. |
||||
7. This feature is an optional extension-level capability; projects that do not use Python syntax do not depend on phpy. |
||||
|
||||
The most important language change is the Python special root namespace. `python\module\member()` in the global namespace, or `\python\module\member()` in other namespaces, can directly access module members; `use python\module` works entirely according to PHP's ordinary namespace alias rules, and the compiler does not apply any Python-specific treatment to the `use` statement. Both forms elevate what phpy originally required writing by hand as `PyCore::import('module')` and a returned variable into a compile-time-recognizable lazy module binding. Capabilities such as attributes, methods, subscripts, iteration, argument conversion, return wrapping, and exceptions for Python objects reuse phpy's existing implementation in principle, without re-establishing a separate runtime in TypePHP. |
||||
|
||||
Non-goals: |
||||
|
||||
- Do not compile Python source code, and do not attempt to replace CPython. |
||||
- Do not promise to statically type the dynamic Python API. |
||||
- Permanently do not support Python threads, `asyncio`, or CPython subinterpreters. |
||||
- Do not generate a Python extension, and do not register TypePHP functions, classes, or modules with Python. |
||||
- Do not provide `#[PythonExport]` or any other TypePHP symbol export mechanism. |
||||
- Do not aim to be compatible with Python syntax; the goal is to let TypePHP programs call Python packages conveniently and reliably. |
||||
- Do not automatically and recursively copy arbitrary Python containers into TypePHP arrays. |
||||
|
||||
## 2. Reference Designs |
||||
|
||||
### 2.1 Mojo |
||||
|
||||
Mojo uses an unmodified CPython runtime to guarantee Python ecosystem compatibility, and wraps dynamic Python values with a unified `PythonObject`. TypePHP only borrows its embedding and object-wrapping design, not its export mechanism. |
||||
|
||||
Borrowable parts: |
||||
|
||||
- Python values remain as wrapper objects by default. |
||||
- TypePHP base values can be automatically converted when passed into Python. |
||||
- Conversion from Python values back to TypePHP native types is explicit. |
||||
- Dynamic Python values are carried by a unified proxy type. |
||||
|
||||
References: [Mojo Python interoperability](https://docs.modular.com/stable/mojo/manual/python/), [Mojo Python types](https://docs.modular.com/mojo/manual/python/types). |
||||
|
||||
### 2.2 pybind11 |
||||
|
||||
pybind11 explicitly distinguishes object ownership, return value policies, interpreter lifetime, GIL guards, positional arguments, and keyword arguments. Its experience shows that the most dangerous part of cross-language calls is not the call syntax, but object lifetime and exception paths. |
||||
|
||||
TypePHP should not expose pybind11's ownership policies to ordinary users, but the runtime must establish an equally strict internal contract. |
||||
|
||||
References: [pybind11 embedding](https://pybind11.readthedocs.io/en/stable/advanced/embedding.html), [pybind11 functions](https://pybind11.readthedocs.io/en/stable/advanced/functions.html). |
||||
|
||||
### 2.3 PyO3 |
||||
|
||||
PyO3 uses GIL tokens and Python object pointers with lifetimes, distinguishing held objects, borrowed objects, and GIL-bound objects at the type system level. |
||||
|
||||
TypePHP does not need to expose lifetime parameters to users, but phpy's C++ layer should borrow from this: every CPython API call must be able to prove the GIL is currently held, and every `PyObject*` must be clearly an owned, borrowed, or stolen reference. |
||||
|
||||
References: [PyO3 object model](https://pyo3.rs/main/doc/pyo3/), [PyO3 Python object types](https://pyo3.rs/main/types). |
||||
|
||||
## 3. The Positioning of phpy |
||||
|
||||
phpy is a runtime foundation candidate for this feature, not a validated stable dependency. |
||||
|
||||
Reusable capabilities include: |
||||
|
||||
- Initializing CPython within the ZendVM process. |
||||
- Boundary conversion between `zval` and `PyObject*`. |
||||
- Proxy objects for Python modules, objects, strings, sequences, dictionaries, sets, iterators, and callables. |
||||
- Callable proxies for TypePHP/PHP closures passed into Python. |
||||
- Basic mapping from Python exceptions to Zend exceptions. |
||||
- Infrastructure for Python synchronous calls to functions, objects, and callable proxies actively passed in by the ZendVM. |
||||
- A prototype of the GIL RAII guard. |
||||
|
||||
However, it cannot be assumed that the existing implementation is completely correct. Subsequent implementation must simultaneously review phpy, refactor the boundaries, add tests, fix bugs, and optimize performance. |
||||
|
||||
Key audit items already identified during the design phase: |
||||
|
||||
- CPython initialization, repeated initialization, shutdown ordering, and destruction of still-alive objects. |
||||
- The owned/borrowed/stolen reference rules for every CPython API. |
||||
- The `Py_INCREF/Py_DECREF` symmetry on all success and exception paths. |
||||
- GIL acquisition, reentrant calls, and the behavior of TypePHP calling Python, which calls back into TypePHP, which calls Python again. |
||||
- The conversion process has been changed so that each top-level conversion creates an independent C++ converter object; the conversion policy, recursion stack, and depth limit are all per-object state, restored by RAII, and no longer use process-level or thread-level temporary function pointers. Cross-VM callbacks and lifetime boundaries still need to continue to be audited. |
||||
- After Python exceptions are converted to Zend exceptions, whether the CPython error indicator is always correctly cleared. |
||||
- When Zend exceptions are converted to Python exceptions, preservation of the original exception type, message, and traceback. |
||||
- Whether the operator protocol is correct. For example, PHP `/` must not be mapped to Python floor division. |
||||
- Python big integers, invalid UTF-8, bytes containing NUL, recursive containers, and cyclic references. |
||||
- When Python proxies hold Zend objects, the cross-VM reference cycles that may form between the Zend GC and the CPython GC. |
||||
- Python threads, `asyncio`, and subinterpreters must be permanently and explicitly rejected, rather than producing undefined behavior. |
||||
|
||||
TypePHP dynamically calls the `PyCore`, `PyObject`, `PyDict`, and other Facades exposed by the phpy extension through the ZendVM, without directly linking `libphpy.so` and without generating any phpy C++ symbol references. Existing public names must be preserved; TypePHP does not establish a second user-visible naming system. |
||||
|
||||
Division of responsibilities: |
||||
|
||||
- phpy is responsible for all runtime concerns: CPython initialization, the GIL, reference counting, object proxies, type conversion, exceptions, and the dual-VM lifecycle. |
||||
- phpy is responsible for providing a stable, testable Zend internal class/function/object-handler API. |
||||
- TypePHP is only responsible for recognizing language syntax, static types, and evaluation order, and generating dynamic calls based on `zend_function*` and the PHPX/Zend generic object API. |
||||
- TypePHP does not directly operate on raw `PyObject*`, and does not duplicate phpy's GIL, reference counting, or exception implementation. |
||||
- When fixing runtime bugs, fix phpy first, rather than only adding patches in TypePHP's generated code to bypass them. |
||||
|
||||
Minimal adaptation principle: |
||||
|
||||
- TypePHP's core new capabilities are Python `use` resolution, module alias symbols, and corresponding code generation. |
||||
- `python\name()`, `module\name`, `module\name()`, and operator lowering should all land on phpy's Zend Facade; Python operators call the complete CPython operator protocol through the standard library `operator` module. |
||||
- For behavior that phpy has already correctly solved, only add tests and reuse it; only modify phpy when review or tests prove a bug, that implicit conversion does not conform to TypePHP rules, or that a Zend dynamic entry point is missing. |
||||
- TypePHP does not implement CPython protocol details, and does not duplicate `PyCore`, `PyObject`, or `PyModule` logic in generated code. |
||||
|
||||
## 4. Optional Extension and Runtime Detection |
||||
|
||||
Python interop is an extension-level feature, not a mandatory dependency of the TypePHP core program. |
||||
|
||||
- TypePHP generated code only depends on ZendVM/PHPX, does not include phpy headers, and does not link `libphpy.so`. |
||||
- The compiler recognizes Python syntax and retains logical `PyObject` type information, but does not check whether the phpy SDK, dynamic library, ABI, or Python module exists. |
||||
- phpy must be loaded by the runtime environment like an ordinary PHP extension and register `PyCore`, `PyObject`, and other Zend internal classes. |
||||
- The first time a Python symbol is actually used, TypePHP resolves `PyCore` and the corresponding `zend_function*` through the class map/func map. |
||||
- When phpy is not loaded, the Zend class lookup throws a catchable PHP `Error`; if not caught, it becomes a fatal error under ordinary PHP rules. |
||||
- When phpy is loaded but the Python module does not exist, `PyCore::import()` throws a `PyError` through phpy. |
||||
- When there is only `use python\sys` without actually accessing any Python symbol, no runtime resolution occurs, so no error is reported even if phpy is not installed. |
||||
|
||||
This model allows the same TypePHP binary to run Python-free paths in environments without phpy installed, and avoids TypePHP establishing a native C++ ABI dependency on phpy. |
||||
|
||||
### 4.1 TypePHP Code Isolation |
||||
|
||||
All Python-specific implementation in TypePHP must be concentrated in an independent subdirectory, tentatively: |
||||
|
||||
```text |
||||
src/Python/ |
||||
``` |
||||
|
||||
This directory is responsible for: |
||||
|
||||
- Recognition of the `python` special root namespace. |
||||
- The import/module symbol table. |
||||
- Python Zend class/method name and logical return type mapping. |
||||
- Python syntactic sugar and static return type mapping. |
||||
- C++ lowering for Python calls, attributes, subscripts, iteration, and operators. |
||||
- Python-specific diagnostics. |
||||
|
||||
The general Parser, TypeSystem, Optimizer, and Generator may only keep minimal, stable extension entry points, and should not scatter `if ($isPython...)` special cases. When the Python feature is not enabled, Python-specific analyzers are not loaded, and existing code generation paths are not changed. |
||||
|
||||
Tests are likewise organized independently; it is recommended to use: |
||||
|
||||
```text |
||||
phpunit/src/Python/ |
||||
phpunit/code/python/ |
||||
tests/compiler/python/ |
||||
``` |
||||
|
||||
The specific directory names will be confirmed during the coding plan phase, but "implementation and test isolation" is a design constraint. |
||||
|
||||
## 5. Overall Runtime Model |
||||
|
||||
The following model is adopted: |
||||
|
||||
- One ZendVM and one CPython main interpreter coexist in one process. |
||||
- CPython is initialized and shut down entirely through phpy's existing extension lifecycle; TypePHP does not establish a second initialization path. |
||||
- All Python API boundaries automatically acquire the GIL; ordinary users do not operate the GIL. |
||||
- `PyObject` and its subclasses such as `PyDict`, `PyList`, and `PyStr` hold CPython strong references. |
||||
- Copying Python proxy objects increments the reference count; destruction decrements it within a valid interpreter/GIL context. |
||||
- Borrowed references are only allowed to exist in phpy's internal short-lifetime scopes and are not exposed to TypePHP. |
||||
- TypePHP calling Python, Python synchronously calling back a callable that TypePHP passed in as an argument, and that callable calling Python again must all support synchronous reentrancy. |
||||
- Python cannot independently import TypePHP applications, nor look up TypePHP functions or types through a global registry. |
||||
|
||||
All Python objects held by TypePHP must be released before the interpreter shuts down. One must not rely on `Py_Finalize()` to automatically repair incorrect lifetimes. |
||||
|
||||
## 6. Module Names and Import Syntax |
||||
|
||||
`python` is a reserved root namespace recognized by the compiler: |
||||
|
||||
```php |
||||
python\math\sqrt(16); |
||||
python\os\path\join('/tmp', 'file.txt'); |
||||
|
||||
use python\sys; |
||||
use Python\numpy as np; |
||||
use python\numpy\linalg as linalg; |
||||
``` |
||||
|
||||
A fully qualified name does not require writing `use` first: |
||||
|
||||
```php |
||||
$root = python\math\sqrt(16); |
||||
$pi = Python\math\pi; |
||||
``` |
||||
|
||||
All segments after the Python root and before the last `\` constitute the Python module path, and the last segment is the module member. PHP's `\` is converted to Python's `.` during import. Therefore `python\os\path\join()` in the global namespace unambiguously means the `join` callable of module `os.path`. |
||||
|
||||
Python module names still strictly obey PHP's namespace resolution rules. When inside an ordinary PHP namespace, a fully qualified module name must use a leading `\`: |
||||
|
||||
```php |
||||
namespace App; |
||||
|
||||
\python\math\sqrt(16); // Python module math |
||||
python\math\sqrt(16); // ordinary PHP name App\python\math\sqrt, not a Python module |
||||
``` |
||||
|
||||
This is part of PHP syntax; `python` is not treated as an exception that bypasses the current namespace. `use python\math;` imports from the root name like any other PHP `use` declaration, so an alias shorthand can also be used inside a namespace. |
||||
|
||||
`use` is only used to shorten a fully qualified name, not a precondition for accessing a Python module: |
||||
|
||||
PHP's `use function` and `use const` apply equally, and ordinary `as` aliases are supported: |
||||
|
||||
```php |
||||
use function python\len; |
||||
use function python\math\sqrt as py_sqrt; |
||||
use const python\math\pi as py_pi; |
||||
|
||||
$length = len([1, 2, 3]); |
||||
$root = py_sqrt(16); |
||||
$pi = py_pi; |
||||
``` |
||||
|
||||
These declarations are still handled entirely by PHP name resolution. TypePHP only enters Python lowering when the final fully qualified name of a `FuncCall` or `ConstFetch` is in the root namespace `python\...`; the `use` declaration itself does not import a Python module. |
||||
|
||||
Respectively equivalent to: |
||||
|
||||
```python |
||||
import sys |
||||
import numpy as np |
||||
import numpy.linalg as linalg |
||||
``` |
||||
|
||||
In the existing phpy PHP API, semantically corresponding to: |
||||
|
||||
```php |
||||
$sys = PyCore::import('sys'); |
||||
$np = PyCore::import('numpy'); |
||||
$linalg = PyCore::import('numpy.linalg'); |
||||
``` |
||||
|
||||
`PyCore::import()` returns a `PyModule`/`PyObject` variable, and subsequent attributes and methods are accessed through that variable. `use python\module` is just an ordinary PHP namespace alias; it does not perform the import immediately, and does not generate a ZendVM class, namespace, or user-visible variable. The compiler only checks the fully qualified name resolved by PHP when processing a function call or constant read. |
||||
|
||||
When the compiler finds `module\attr` or `module\func()` in function code, it uses the same compiler structure as the existing `funcMap`: it allocates an integer ID for the actually-used fully qualified module name, generates a unified `THREAD_LOCAL` zval array, and dynamically calls `PyCore::import()` through a lazy getter. The following names are only design sketches: |
||||
|
||||
```cpp |
||||
THREAD_LOCAL zval php_python_module_map[module_count]; |
||||
|
||||
php::Object php_get_python_module(int module_id, const php::Str &module_name) |
||||
{ |
||||
zval *module = &php_python_module_map[module_id]; |
||||
if (UNEXPECTED(Z_ISUNDEF_P(module))) { |
||||
// Resolve PyCore::import through classMap/funcMap and invoke zend_function*. |
||||
php::Variant value = php::call(/* cached zend_function* */, php::ArgList{module_name}); |
||||
ZVAL_COPY(module, value.ptr()); |
||||
} |
||||
return php::Object(module); |
||||
} |
||||
``` |
||||
|
||||
Corresponding lowering: |
||||
|
||||
```text |
||||
use Python\numpy as np |
||||
-> compile-time namespace marker: np => "numpy" |
||||
-> module id allocated only when np is actually referenced |
||||
|
||||
np\version |
||||
-> php::Object(php_get_python_module(module_id, "numpy")).attr("version") |
||||
|
||||
np\array($value) |
||||
-> php::Object(php_get_python_module(module_id, "numpy")).call("array", converted($value)) |
||||
|
||||
python\numpy\array($value) |
||||
-> the same module id and lowering as np\array($value) |
||||
|
||||
python\os\path\join($left, $right) |
||||
-> php::Object(php_get_python_module(module_id, "os.path")).call("join", ...) |
||||
``` |
||||
|
||||
The same fully qualified module name is allocated only one ID across the entire TypePHP build; when the fully qualified name and any `use` alias refer to the same module, they also share that ID. If the current `.php` file only has `use python\sys` but no `sys\attr`, `sys\func()`, or other `sys` symbol access appears, the compiler does not allocate a module ID for it, the runtime does not call `import('sys')`, and no error is reported because the Python environment lacks that module. |
||||
|
||||
An unused module does not trigger any phpy runtime resolution. `tpc` only checks the syntax and alias conflicts of `use python\sys` itself, does not check the phpy SDK/ABI, and does not add phpy link dependencies. |
||||
|
||||
### 6.1 Relationship with `funcMap` |
||||
|
||||
`pythonModuleMap` reuses the overall validated pattern of `funcMap`: |
||||
|
||||
- At compile time, a `fully qualified module name -> integer ID` map is used for deduplication. |
||||
- Data declarations are generated centrally; ordinary `.cc` files only reference the extern array and getter. |
||||
- The getter initializes on first access, and subsequent accesses hit the array directly. |
||||
- IDs are only allocated for modules where member access or calls actually occur. |
||||
- Cleanup is centralized during the application/request clean phase. |
||||
|
||||
However, the two cannot mechanically use the exact same cleanup code: |
||||
|
||||
- `funcMap` stores non-owning `zend_function*` owned by the Zend function table, so it can be cleaned up directly with `memset`. |
||||
- `pythonModuleMap` stores Zend `PyModule` object zvals returned by phpy, and cannot directly `memset` over valid objects. |
||||
- Request clean must perform `zval_ptr_dtor()` on each entry and restore it to `UNDEF`, letting phpy's own Zend object destructor handle Python references and the GIL. |
||||
- On import failure, the slot stays `UNDEF`; exception values or half-initialized objects must not be cached. |
||||
|
||||
Cleanup is done by TypePHP using the ordinary Zend zval API, without calling phpy C++ symbols: |
||||
|
||||
```cpp |
||||
for (zval &module : php_python_module_map) { |
||||
if (!Z_ISUNDEF(module)) { |
||||
zval_ptr_dtor(&module); |
||||
ZVAL_UNDEF(&module); |
||||
} |
||||
} |
||||
``` |
||||
|
||||
TypePHP only releases the Zend object; its internal Python reference counting, GIL, and error state are still handled by the phpy object handler. |
||||
|
||||
### 6.2 `sys.modules` Remains the Global Source of Truth |
||||
|
||||
Python import is itself global. When the getter first calls the underlying import, CPython returns the already-loaded module from `sys.modules` or performs the first load. `pythonModuleMap` is not a second import system; it is only equivalent to the binding stored in a Python file's namespace after it executes `import numpy as np`: |
||||
|
||||
```text |
||||
php_get_python_module(id, "numpy") |
||||
-> PyModule zval binding within the TypePHP request |
||||
-> CPython sys.modules (global module identity and load state) |
||||
``` |
||||
|
||||
It avoids re-entering the Python import API on every function call, while not taking on package lookup, loading, or reload logic. Even if the same module is referenced by multiple TypePHP files under different aliases, as long as the fully qualified module name is the same, the same ID and `PyModule` Zend object zval are used. |
||||
|
||||
This binding is consistent with ordinary Python import: if Python code later deletes or replaces `sys.modules['numpy']`, the already-completed `np` binding is not automatically changed; an explicit `PyCore::import('numpy')` is handled according to the `sys.modules` state at the time of the call. |
||||
|
||||
Rules: |
||||
|
||||
- The `python` root namespace is case-insensitive; `python`, `Python`, and `PYTHON` are all recognized as the same language symbol. |
||||
- Only the root namespace is case-insensitive. Subsequent module paths, members, methods, and keyword argument names are strictly case-sensitive. |
||||
- `python\package\module\member` in the global namespace, and `\python\package\module\member` in other namespaces, are fully qualified module accesses that do not require `use` and perform lazy import on first actual execution. |
||||
- `python\...` without a leading `\` inside a namespace is a relative PHP name and must have the current namespace prepended under PHP rules; it cannot be recognized as a Python module. |
||||
- `use python\...` can only import Python modules. |
||||
- Whether the module exists can only be determined by CPython at runtime. |
||||
- `from package import *` is not supported. |
||||
- The first version does not design a separate `from package import name` syntax; members are uniformly accessed through module aliases. |
||||
- The root namespace `\python` is reserved for language interop; for example, `App\python` is still an ordinary PHP namespace. |
||||
- Module aliases must not conflict with TypePHP classes, namespace imports, or other Python module aliases in the current file. |
||||
- Users can still call `PyCore::import()` directly and save the returned `PyModule` into an ordinary variable; fully qualified names and names resolved through ordinary PHP `use` both use the `pythonModuleMap` lazy binding. |
||||
|
||||
Example: |
||||
|
||||
```php |
||||
python\len($value); // correct |
||||
Python\len($value); // correct, different root namespace casing |
||||
python\Len($value); // error, wrong casing for the Python builtin name |
||||
Python\Len($value); // error |
||||
``` |
||||
|
||||
When the resolution result is in the root namespace `\python`, it is converted by the TypePHP compiler into a Python language symbol; when it resolves to another name such as `App\python`, an ordinary PHP function or class lookup is still performed. |
||||
|
||||
## 7. Module Members |
||||
|
||||
A Python module appears in TypePHP as a namespace, not a class. Names in the module are still dynamically resolved by the Python VM as attributes. |
||||
|
||||
### 7.1 Package Variables |
||||
|
||||
Reading a Python package variable uses the PHP namespace constant syntax `module\name`: |
||||
|
||||
```php |
||||
use python\math; |
||||
use python\os; |
||||
use python\numpy as np; |
||||
|
||||
$pi = math\pi; |
||||
$environ = os\environ; |
||||
$arrayType = np\ndarray; |
||||
$directPi = python\math\pi; |
||||
$text = math\pi->__str__(); |
||||
``` |
||||
|
||||
This uses PHP's legal namespace constant expression, but TypePHP does not register it as a Zend constant and does not perform constant folding. The compiler lowers each read to a Python module attribute lookup, and the result remains a `PyObject`, so object methods can continue to be called. |
||||
|
||||
`math::pi` or `math::$pi` is not allowed for reading package variables; both are class member syntax and would incorrectly express the module as a class. When the compiler finds such legacy syntax, it produces a targeted FatalError and suggests using `math\pi` instead. |
||||
|
||||
### 7.2 Package Functions and Class Construction |
||||
|
||||
Calling a callable in a Python package uses the PHP namespace function syntax `module\name(...)`: |
||||
|
||||
```php |
||||
$a = np\array([1, 2, 3]); |
||||
$b = np\array([4, 5, 6]); |
||||
$c = np\add($a, $b); |
||||
$root = python\math\sqrt(16); |
||||
$joined = python\os\path\join('/tmp', 'file.txt'); |
||||
``` |
||||
|
||||
The compiler reads the module's `name` attribute and calls the resulting Python object. That object can be: |
||||
|
||||
- A Python function. |
||||
- A Python class, in which case the call performs that class's construction process and returns an instance. |
||||
- Another Python object implementing `__call__`. |
||||
|
||||
TypePHP does not need to, and cannot, determine from the `np\array()` syntax alone whether it is a function or a class construction; callability is determined by Python at runtime. When a member does not exist, a Python `AttributeError` is produced; when a member is not callable, a Python `TypeError` is produced; both are uniformly mapped to `PyError`. |
||||
|
||||
The first version only supports reading module attributes. PHP namespace constant syntax itself cannot be an assignment target; when writing is needed, it should be done explicitly through the Python object API: |
||||
|
||||
```php |
||||
$os = PyCore::import('os'); |
||||
python\setattr($os, 'name', $value); |
||||
``` |
||||
|
||||
## 8. Python Builtins and phpy Syntactic Sugar |
||||
|
||||
`python\name()` means calling a Python builtin: |
||||
|
||||
```php |
||||
python\print('hello'); // equivalent to PyCore::print('hello') |
||||
$length = python\len($value)->toValue()->toInt(); |
||||
$range = python\range(0, 10); |
||||
$type = python\type($value); |
||||
``` |
||||
|
||||
It is not an ordinary TypePHP namespace function. The compiler resolves the `zend_function*` corresponding to `PyCore` through the class/func map and dynamically calls it; the runtime semantics are consistent with directly writing the corresponding `PyCore` call. |
||||
|
||||
Names are strictly case-sensitive. For known wrong names in the compiler's builtin mapping, errors can be reported at compile time; failures of other dynamic builtin lookups produce a Python `AttributeError`. |
||||
|
||||
Some names are syntactic sugar for existing phpy type constructors, rather than directly calling the same-named Python builtin: |
||||
|
||||
| TypePHP syntax | Equivalent phpy API | |
||||
|---|---| |
||||
| `python\dict($array)` | `new PyDict($array)` | |
||||
| `python\list($array)` | `new PyList($array)` | |
||||
| `python\tuple($array)` | `new PyTuple($array)` | |
||||
| `python\set($array)` | `new PySet($array)` | |
||||
| `python\str($value)` | `new PyStr($value)` | |
||||
| `python\object($value)` | `new PyObject($value)` | |
||||
| `python\print(...)` | `PyCore::print(...)` | |
||||
| `python\scalar($value)` | `PyCore::scalar($value)` | |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
$dict1 = new PyDict([1, 2, 3, 4]); |
||||
$dict2 = python\dict([1, 2, 3, 4]); |
||||
``` |
||||
|
||||
The two must have exactly the same runtime semantics. Here one cannot simply forward to CPython's `dict([1, 2, 3, 4])`, because the native Python builtin would interpret the argument as a key/value pair iterable, which differs from `PyDict`'s PHP array construction rules. |
||||
|
||||
The mapping of all syntactic sugar must form a closed, tested table; it must not be guessed based on function names alone. |
||||
|
||||
This mapping also determines compile-time static types: |
||||
|
||||
```php |
||||
$list1 = new PyList(); |
||||
$list2 = python\list(); |
||||
|
||||
$dict1 = new PyDict(); |
||||
$dict2 = python\dict(); |
||||
``` |
||||
|
||||
- Both `$list1` and `$list2` are `PyList` typed objects. |
||||
- Both `$dict1` and `$dict2` are `PyDict` typed objects. |
||||
- Both forms must use the same type checking, method resolution, and Native Call optimization. |
||||
- Syntactic sugar must not degrade to `mixed`, `var`, or only the base type `PyObject`. |
||||
- Python builtin calls also obey the object-retention rule. For example, `python\len()` returns a `PyObject` wrapping a Python int; to obtain a definite type, one must first leave the Python object rules via `toValue()` (or the function entry `python\scalar()`), then use ordinary TypePHP conversion. The Python `None` result of `python\print()` also remains a `PyObject`; when used as a standalone statement, it can simply be discarded. |
||||
- Neither `PyObject::toValue()` nor `python\scalar()` is an ordinary Python builtin call; both are conversion boundaries that explicitly require exiting the Python type rules, so they return a TypePHP `var`. |
||||
- Dynamic Python module member calls uniformly return `PyObject`. |
||||
|
||||
## 9. Python Object Types |
||||
|
||||
All Python values whose static type cannot be determined at compile time are uniformly represented as: |
||||
|
||||
```php |
||||
PyObject |
||||
``` |
||||
|
||||
`PyObject` is phpy's existing public type and TypePHP's official runtime type. `python\Object` or `python\Any` will not be introduced. |
||||
|
||||
Python built-in types continue to use phpy's existing concrete proxy classes, such as `PyDict`, `PyList`, `PyTuple`, `PySet`, `PyStr`, `PyType`, `PyFn`, and `PyIter`. In this way, ordinary PHP and TypePHP users see the same type system. |
||||
|
||||
Python's `None` is also a legal Python object. Its automatic conversion rules with TypePHP `null` need to be defined separately; Python `None` cannot be represented by a null pointer. |
||||
|
||||
## 10. Object Operations |
||||
|
||||
### 10.1 Attributes and Methods |
||||
|
||||
```php |
||||
$env = os\environ; |
||||
$items = $env->items(); |
||||
$name = $object->name; |
||||
$object->name = 'new value'; |
||||
unset($object->name); |
||||
``` |
||||
|
||||
These are respectively mapped to Python's `getattr`, call, `setattr`, and `delattr` protocols. |
||||
|
||||
`PyObject` explicitly provides two PHP Facade methods, `toValue()` and `toArray()`. `toValue()` is equivalent to `PyCore::scalar()` / `python\scalar()`, recursively converting a Python value into a PHP builtin value. Its return value then uses ordinary TypePHP conversion methods to determine the type: |
||||
|
||||
```php |
||||
$pyValue = np\int64(42); // PyObject |
||||
$value = $pyValue->toValue()->toInt(); // TypePHP int |
||||
``` |
||||
|
||||
Here `toInt()` acts on the TypePHP value already returned by `toValue()`, not on the `PyObject`. |
||||
|
||||
`toArray()` only converts Python `list`, `tuple`, `set`, `dict`, and iterators. Container elements are recursively converted to PHP values; an iterator is consumed, and subsequent conversion can only obtain its remaining elements. Python types that are not convertible return an empty array. `toArray()` is also a TypePHP keyword method, but PHPX's object conversion path calls `PyObject::toArray()`; `toString()` continues to call `PyObject::__toString()` through the keyword method, and phpy does not declare `toString()` again. |
||||
|
||||
### 10.2 Subscripts |
||||
|
||||
```php |
||||
$value = $object[$key]; |
||||
$object[$key] = $value; |
||||
unset($object[$key]); |
||||
isset($object[$key]); |
||||
``` |
||||
|
||||
These are respectively mapped to the Python mapping/sequence protocol. |
||||
|
||||
`isset()` keeps PHP's emptiness semantics: it returns `false` when the key or index does not exist, and also returns `false` when the corresponding value is Python `None`. The runtime only recognizes `KeyError` / `IndexError` as "missing"; other exceptions thrown by the Python protocol must continue to be mapped to `PyError` and must not be swallowed by `isset()`. Integer subscripts of list and tuple follow Python's negative index rules. |
||||
|
||||
### 10.3 Calling Objects |
||||
|
||||
```php |
||||
$result = $callable($arg1, $arg2); |
||||
``` |
||||
|
||||
The runtime uses `PyObject_Call`. A non-callable object produces a Python `TypeError`, mapped to a TypePHP-catchable Python exception. |
||||
|
||||
### 10.4 Iteration |
||||
|
||||
```php |
||||
foreach ($pythonIterable as $value) { |
||||
// Python __iter__ / __next__ |
||||
} |
||||
``` |
||||
|
||||
The keyed form: |
||||
|
||||
```php |
||||
foreach ($pythonIterable as $index => $value) { |
||||
} |
||||
``` |
||||
|
||||
A generic Python iterator uses the TypePHP iteration ordinal starting from `0` as `$index`, and `$value` is the object produced by `__next__()`. `PyDict` is phpy's dedicated mapping wrapper; a keyed `foreach` uses PHP mapping conventions: `$index` is the dict key, and `$value` is the corresponding dict value. Python exceptions from `__iter__()` / `__next__()` must be propagated as `PyError` and must not be treated as normal end-of-iteration. |
||||
|
||||
## 11. Arguments and Keyword Arguments |
||||
|
||||
Ordinary arguments are evaluated from left to right, then Python positional args are constructed: |
||||
|
||||
```php |
||||
$model = AutoModel\from_pretrained( |
||||
'model-name', |
||||
trust_remote_code: true, |
||||
device_map: 'auto', |
||||
); |
||||
``` |
||||
|
||||
TypePHP named arguments are mapped to Python keyword arguments. Argument names are strictly case-sensitive. |
||||
|
||||
PHP/TypePHP array unpacking rules can be used to construct positional and keyword arguments, but must satisfy: |
||||
|
||||
- Integer keys produce positional arguments. |
||||
- String keys produce keyword arguments. |
||||
- Positional arguments must not appear after keyword arguments. |
||||
- Duplicate keywords produce a Python `TypeError`. |
||||
|
||||
Whether to add explicit `python\args()` / `python\kwargs()` types is left for later discussion; the first version reuses the existing call and array unpacking syntax as much as possible. |
||||
|
||||
## 12. Explicit Conversion Principle |
||||
|
||||
TypePHP does not inherit phpy's return value implicit conversion behavior at the ZendVM Facade/opcode level. At the language level it adopts the principle of "automatic conversion when arguments enter the Python boundary, Python return values remain objects, and explicit conversion when returning to TypePHP." |
||||
|
||||
Scenarios where automatic conversion is allowed must be syntactically explicit that the code is entering Python: |
||||
|
||||
- `python\name(...)`. |
||||
- Python module calls, such as `np\array(...)`. |
||||
- Calls to `PyObject` methods or callables. |
||||
- Explicit Python container construction, such as `new PyList(...)` or `python\list(...)`. |
||||
- Parameter declarations requiring `PyObject`, `PyDict`, or other phpy types. |
||||
- Mixed arithmetic expressions composed of `PyObject` and TypePHP values. |
||||
|
||||
Within these call boundaries, all argument expressions are first strictly evaluated in TypePHP left-to-right order, then converted into objects that Python can accept. TypePHP scalars are converted to the corresponding Python scalars; TypePHP arrays are recursively converted to Python list/dict, a process that produces deep copies. This must not spread into a global implicit conversion in ordinary TypePHP expressions that do not contain Python objects. |
||||
|
||||
"Automatic conversion of all arguments" only applies to TypePHP types explicitly supported by the conversion table; resource or other values without a Python representation must throw a clear type error, not silently convert or pass invalid pointers. |
||||
|
||||
Implicit conversion is not allowed in the following scenarios: |
||||
|
||||
- Assigning a `PyObject` directly to `int`, `float`, `bool`, `string`, or `array`. |
||||
- Implicitly deep-copying a Python container into a TypePHP array. |
||||
- Arbitrarily turning a Python object into a TypePHP scalar due to arithmetic, comparison, or string context. |
||||
- Changing the static type of a TypePHP variable based on the runtime Python type. |
||||
|
||||
`echo $pyObject` can continue to be compatible with the existing `PyObject::__toString()`, but this only belongs to the output protocol and cannot be treated by the compiler as a general string implicit conversion. |
||||
|
||||
## 13. TypePHP to Python Conversion |
||||
|
||||
The Python call boundary allows the following automatic conversions: |
||||
|
||||
| TypePHP | Python | Semantics | |
||||
|---|---|---| |
||||
| `null` | `None` | singleton, not an empty `PyObject*` | |
||||
| `bool` | `bool` | value conversion | |
||||
| `int` | `int` | Python arbitrary-precision integer | |
||||
| `float` | `float` | double | |
||||
| `string` | `str` | requires valid UTF-8 | |
||||
| list array | `list` | recursive copy | |
||||
| map array | `dict` | recursive copy | |
||||
| `PyObject` and subclasses | the original object | zero copy, only passing a reference | |
||||
| TypePHP callable | Python callable proxy | Python can synchronously call back TypePHP | |
||||
| TypePHP object | Zend object proxy | object attributes are not automatically copied | |
||||
|
||||
PHP arrays use rules such as `zend_array_is_list()` to decide whether to convert to a Python `list` or `dict`. An empty array defaults to a Python `list`; if an empty dict is needed, an explicit construction API must be provided. |
||||
|
||||
Arrays and ordinary TypePHP strings may incur allocation and copying each time they enter the Python boundary. Documentation and performance diagnostics should suggest constructing and reusing native Python proxy types such as `PyDict`, `PyList`, and `PyStr` early for high-frequency calls, loop calls, or large-data scenarios, to avoid repeated deep copies. When `PyObject` and its subclasses enter the Python boundary, only a reference to the original object is passed; no content copying occurs. |
||||
|
||||
Recommended form: |
||||
|
||||
```php |
||||
// Convert only once; subsequent calls pass the same Python object. |
||||
use python\processor; |
||||
|
||||
$pyItems = python\list($items); |
||||
for ($i = 0; $i < 1000; $i++) { |
||||
processor\consume($pyItems); |
||||
} |
||||
``` |
||||
|
||||
Passing the same TypePHP container repeatedly as an argument inside a loop should be avoided, because each crossing of the Python call boundary deep-copies it again: |
||||
|
||||
```php |
||||
for ($i = 0; $i < 1000; $i++) { |
||||
processor\consume($items); |
||||
} |
||||
``` |
||||
|
||||
Strings and bytes must be distinguished. TypePHP `string` maps to Python `str` by default; binary content uses explicit `python\bytes()`. |
||||
|
||||
Recursive arrays, cyclic references, and overly deep nesting must be detected and throw exceptions, rather than recursing infinitely. |
||||
|
||||
## 14. Python to TypePHP Conversion |
||||
|
||||
### 14.1 Default Rules |
||||
|
||||
TypePHP's Python-specific call paths must turn off phpy's return value implicit conversion; all Python function, method, construction call, and operation results remain phpy objects. The static return type of dynamic calls is uniformly `PyObject`; it must not implicitly convert to a TypePHP value just because the runtime result happens to be a Python `bool`, `int`, `float`, `str`, `list`, or `dict`. |
||||
|
||||
The current implementation has the generated code dynamically call `PyCore::setOptions(['return_as_object' => true])` the first time a Python expression is actually executed. This initialization is a request-level lazy guard: only writing `use python\module` without accessing Python symbols does not trigger phpy; a constructor-only program also completes configuration before construction; request clean resets TypePHP's own guard. If phpy later provides an object-retention standalone entry point without global mode, this runtime implementation can be replaced without changing the language semantics. |
||||
|
||||
The phpy construction syntactic sugar known to the compiler still retains the precise subclasses; for example, `python\list()` returns `PyList` and `python\dict()` returns `PyDict`; these types are all `PyObject` subclasses and do not constitute return value implicit conversion. |
||||
|
||||
The phpy Zend Facade should provide mutually independent "retain Python object" and "explicitly convert to TypePHP" entry points. It must not temporarily switch by modifying process-level global function pointers or global conversion modes, otherwise nested calls, synchronous reentrancy, and exception paths may leak the wrong policy to subsequent calls. Ordinary Python calls generated by TypePHP only dynamically call the object-retention entry point; `PyObject::toValue()` and `python\scalar()` ultimately call the explicit scalar conversion entry point. |
||||
|
||||
phpy internally already uses `PythonToPhpConverter` and `PhpToPythonConverter` to implement this constraint. Each top-level conversion has an independent instance, and recursive sub-values reuse the same instance; container entry and exit are managed by RAII guards, and cyclic containers or inputs exceeding the depth limit throw a `PyError` without polluting subsequent conversions or causing a process crash. |
||||
|
||||
Reasons: |
||||
|
||||
- Preserve Python object identity and precise type. |
||||
- Avoid immediate deep copying when containers are returned. |
||||
- A Python `int` may exceed the TypePHP `int` range. |
||||
- Subclasses of Python types may override protocols and must not be forcibly expanded as base containers. |
||||
- Avoid phpy's current "partially auto-converted scalars, partially wrapped objects" behavior entering the TypePHP static type system. |
||||
|
||||
### 14.2 Explicit Conversion |
||||
|
||||
A Python object can only enter the TypePHP type rules through `toValue()`, `python\scalar()` (or the equivalent hand-written `PyCore::scalar()`): |
||||
|
||||
```php |
||||
$nativeValue1 = PyCore::scalar($value); |
||||
$nativeValue2 = python\scalar($value); // fully equivalent syntactic sugar |
||||
$nativeValue3 = $value->toValue(); |
||||
$integer = $value->toValue()->toInt(); |
||||
$float = $value->toValue()->toFloat(); |
||||
$boolean = $value->toValue()->toBool(); |
||||
$string = $value->toValue()->toString(); |
||||
$array = $value->toArray(); |
||||
``` |
||||
|
||||
Rules: |
||||
|
||||
- `toValue()` is an ordinary public method of `PyObject`, not registered as a TypePHP keyword method; internally it reuses the same converter as `PyCore::scalar()` in phpy. |
||||
- `toArray()` retains TypePHP global keyword method semantics. When PHPX performs array conversion on an object, it preferentially calls its public `toArray()`, so it enters the phpy implementation. |
||||
- After explicit conversion completes, the result fully enters TypePHP's static type, operator, and argument passing rules, no longer using Python protocols. |
||||
- Container conversion is an explicit deep conversion and detects recursive references. |
||||
- Python big integers must not silently overflow; the existing conversion rules need review before the precise mapping to TypePHP `BigInt` is determined. |
||||
- Python `str` and `bytes` must be distinguished; both must not be unconditionally converted to a TypePHP string. |
||||
- phpy is responsible for the general value conversion of `PyObject::toValue()` / `PyCore::scalar()`, and the limited container conversion of `PyObject::toArray()`. `toInt/toFloat/toBool` are post-conversion PHP values; `toString()` still calls `PyObject::__toString()` through the TypePHP keyword method. |
||||
|
||||
Existing phpy PHP users can retain compatible behavior; TypePHP calls phpy's object-retention Zend API. For this, phpy internal class methods can be refactored or added, but no C++ link dependency from TypePHP to phpy is introduced. |
||||
|
||||
## 15. Operators via the Python `operator` Module |
||||
|
||||
For `PyObject` and its subclasses: |
||||
|
||||
- `+ - * / % ** << >> & | ^` map to the corresponding functions of the Python standard library `operator` module. |
||||
- `/` maps to `operator.truediv()`, and must not map to `operator.floordiv()`. |
||||
- Python floor division temporarily uses `python\floordiv($a, $b)`, because TypePHP has no `//` operator. |
||||
- `== != < <= > >=` map to `operator.eq/ne/lt/le/gt/ge()` respectively. |
||||
- `===` / `!==` map to `operator.is_()` / `operator.is_not()` respectively. |
||||
- `if ($object)`, `!$object` use `operator.truth()`. |
||||
- Compound assignments map to `operator.iadd/isub/...()`, and the returned object updates the lvalue. |
||||
|
||||
Base mapping: |
||||
|
||||
| TypePHP | Generated dynamic call | |
||||
|---|---| |
||||
| `$a + $b` | `operator\add($a, $b)` | |
||||
| `$a - $b` | `operator\sub($a, $b)` | |
||||
| `$a * $b` | `operator\mul($a, $b)` | |
||||
| `$a / $b` | `operator\truediv($a, $b)` | |
||||
| `$a % $b` | `operator\mod($a, $b)` | |
||||
| `$a ** $b` | `operator\pow($a, $b)` | |
||||
| `$a << $b` | `operator\lshift($a, $b)` | |
||||
| `$a >> $b` | `operator\rshift($a, $b)` | |
||||
| `$a & $b` | `operator\and_($a, $b)` | |
||||
| bitwise OR | `operator\or_($a, $b)` | |
||||
| `$a ^ $b` | `operator\xor($a, $b)` | |
||||
| `-$a` | `operator\neg($a)` | |
||||
| `+$a` | `operator\pos($a)` | |
||||
| `~$a` | `operator\invert($a)` | |
||||
| `$a += $b` | `$a = operator\iadd($a, $b)` | |
||||
|
||||
All operands must be evaluated strictly from left to right. |
||||
|
||||
Even if the source code does not explicitly write `use python\operator`, when a Python operator appears, the compiler treats it as an implicit module binding used only for internal lowering, and obtains the `operator` module through the same `pythonModuleMap`. It does not inject a visible alias into the user file, so it does not conflict with the user's own `operator` class or use alias. When the user explicitly writes `use python\operator`, internal lowering and user access reuse the same module ID. |
||||
|
||||
Identity comparison calls `operator\is_()` / `operator\is_not()`. Even if the `operator\eq()` result of two objects is true, as long as they are not the same Python object, `===` is still false. |
||||
|
||||
Mixed arithmetic between Python objects and TypePHP values is allowed. As long as one side of the current operation node has static type `PyObject` or its subclass, the TypePHP expression on the other side is first fully evaluated under TypePHP rules, then the resulting value is converted to a Python object, and finally CPython executes the protocol corresponding to the current operation node. |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
$result1 = $pyInt + 10; // 10 is converted to a Python int; Python performs the addition |
||||
$result2 = $pyList * getCount(); // getCount() is evaluated first, then converted to a Python int |
||||
$native = $pyInt->toValue()->toInt() + 10; // already explicitly converted to a TypePHP int; TypePHP addition is used |
||||
``` |
||||
|
||||
The result of an `operator` call is still a `PyObject`, to preserve arbitrary objects that Python custom operators may return. `===` / `!==` and conditional branches are exceptions: the Python bool results of `operator.is_/is_not/truth()` are subsequently converted through the explicit phpy conversion entry to a TypePHP `bool`. Both operands must be strictly evaluated once each from left to right, and the conversion process must not cause the expression to be executed repeatedly. |
||||
|
||||
When phpy runs as an ordinary PHP extension, it can continue to use Zend opcode handlers to provide operator overloading compatibility; TypePHP does not depend on these handlers. |
||||
|
||||
Dynamic ZendVM code has one explicitly reserved limitation: Zend compiles `-$value` / `+$value` into multiplication by `-1` / `1`, so phpy's opcode handler can no longer recognize the unary operation in the source. Therefore dynamic code retains the `$value * -1` / `$value * 1` protocol behavior and does not rewrite ordinary PHP code through a global AST hook; when a custom Python object's `__neg__()` / `__pos__()` differs from its `__mul__()`, the result may differ. TypePHP AOT still generates `operator.neg()` / `operator.pos()` according to the table above. The external user document `python.md` has already stated this limitation. |
||||
|
||||
When the TypePHP compiler recognizes static types such as `PyObject`, `PyDict`, and other phpy objects, it rewrites operators into ordinary Python module callable calls: |
||||
|
||||
```text |
||||
TypePHP operator |
||||
-> compile-time lowering |
||||
-> implicit python\operator module binding |
||||
-> operator\add/sub/... dynamic call |
||||
-> CPython complete operator protocol |
||||
``` |
||||
|
||||
This abstraction does not directly link phpy and does not go through phpy's user opcode handler, but it is not a zero-cost C++ inline operation: |
||||
|
||||
- `zend_function*` and class entries use the existing func/class map lazy cache. |
||||
- Arguments still need to be constructed as Zend values and converted by phpy into Python objects. |
||||
- Python module member lookup, GIL, CPython call, and reference counting costs still exist. |
||||
- The advantage is that the TypePHP binary only depends on ZendVM/PHPX, and phpy can be a truly optional runtime extension. |
||||
|
||||
Using the standard library `operator.add()` instead of directly calling `__add__()` reuses CPython's complete rules for `NotImplemented`, `__radd__()`, right-operand subclass priority, and so on; TypePHP does not implement reflected-operation fallback. |
||||
|
||||
The current implementation already covers binary arithmetic and bitwise operations, comparison, identity, unary operations, conditional truthiness, short-circuit logic, and compound assignment for variable, attribute, and subscript lvalues. The results of Python module functions/properties, builtins, dynamic methods, attributes, subscripts, and callables all continue to propagate the `PyObject` static type, so they can be chained or participate in subsequent Python operations. |
||||
|
||||
## 16. Exceptions |
||||
|
||||
When a Python call fails, a unified TypePHP exception type is thrown, tentatively: |
||||
|
||||
```php |
||||
PyError |
||||
``` |
||||
|
||||
The exception preserves at least: |
||||
|
||||
- The Python exception type. |
||||
- The message. |
||||
- The Python traceback object. |
||||
- The formatted traceback string. |
||||
- The original Python exception instance. |
||||
|
||||
Example: |
||||
|
||||
```php |
||||
try { |
||||
np\array('invalid')->reshape(2, 2); |
||||
} catch (PyError $error) { |
||||
echo $error->pythonType(); |
||||
echo $error->pythonTraceback(); |
||||
} |
||||
``` |
||||
|
||||
When Python synchronously calls a TypePHP callable proxy and TypePHP throws an exception, it should be converted into an ordinary Python exception, preserving the original TypePHP class name and message. That exception only propagates along the current dynamic call stack; registering a `typephp` Python module or a dedicated global exception type is not required. |
||||
|
||||
After an exception crosses a VM, the source VM's pending exception state must be cleared. Any exception conversion failure must not cause a coredump, duplicate throw, or leftover error state. |
||||
|
||||
## 17. Passing TypePHP Callables to Python |
||||
|
||||
TypePHP functions, closures, and callable objects can be automatically wrapped as Python callables: |
||||
|
||||
```php |
||||
$values = python\list([1, 2, 3]); |
||||
$result = python\map(fn (int $value): int => $value * 2, $values); |
||||
``` |
||||
|
||||
When Python calls the proxy: |
||||
|
||||
1. Python arguments are converted or wrapped into TypePHP values according to boundary rules. |
||||
2. Enter the ZendVM to call the callable. |
||||
3. The return value is converted into a Python value. |
||||
4. TypePHP exceptions are converted into Python exceptions. |
||||
|
||||
A closure proxy must hold the Zend callable to prevent it from being released while Python still references it. Cross-VM reference cycles must be explicitly detected by the runtime or provided with a predictable collection policy. |
||||
|
||||
A TypePHP callable proxy is only an argument value, not an export mechanism: only after TypePHP actively passes the proxy to Python can Python dynamically call it during the object's lifetime. TypePHP does not generate a module that Python can independently import, nor register global functions or classes. |
||||
|
||||
## 18. phpy Lifecycle and Integration |
||||
|
||||
TypePHP reuses phpy's own PHP extension entry points and lifecycle, and does not add an independent CPython bootstrap: |
||||
|
||||
1. phpy's `MINIT` initializes the shared runtime, CPython, and the `PyObject`, `PyDict`, and other Zend classes. |
||||
2. phpy's `RINIT` establishes the state needed for the current request. |
||||
3. During the request, the TypePHP program dynamically calls Python through the internal classes, methods, and object handlers that phpy registers with the ZendVM. |
||||
4. phpy's `RSHUTDOWN` releases request-level resources and proxies. |
||||
5. phpy's `MSHUTDOWN` shuts down the shared runtime and CPython after all proxies have been safely released. |
||||
|
||||
TypePHP should execute these entry points through the same mechanism as other statically or dynamically linked PHP extensions; it must not reinitialize CPython, and must not bypass phpy's lifecycle to directly call `Py_Initialize()` or `Py_Finalize()`. |
||||
|
||||
The only artifact is the main program or library with TypePHP as its entry point. No `.so` / `.pyd` importable by CPython is generated, no TypePHP modules, functions, or classes are registered with Python, and there is no `#[PythonExport]`. |
||||
|
||||
## 19. Performance Principles |
||||
|
||||
- Passing a `PyObject` only adds the necessary reference count and does not copy the Python object. |
||||
- `pythonModuleMap` only caches the `PyModule` Zend object zval that has already been bound; real loading and global identity directly reuse CPython `sys.modules`; builtin/member lookup remains simple in the first version, and a dedicated cache is only designed when benchmarks prove it necessary. |
||||
- Arguments should directly construct the array required by vectorcall, preferring the CPython vectorcall API. |
||||
- Avoid first constructing a PHP array and then having phpy convert it a second time into a Python tuple/dict. |
||||
- Converting a TypePHP array to a Python container is an explicit O(n) conversion and is not claimed to be zero-cost. |
||||
- TypePHP arrays and strings entering hot Python calls should be promoted to reusable `PyList`, `PyDict`, `PyStr`; the compiler does not cache conversion results on its own, because the original TypePHP value may have changed. |
||||
- The GIL guard should cover the minimum necessary region; correct interpreter state must be maintained during single-threaded synchronous reentrancy. |
||||
- Exception paths and normal paths must be equally tested for reference counting and memory leaks. |
||||
|
||||
## 20. Permanent Boundaries and Unsupported Capabilities |
||||
|
||||
- Python threads, including threads created by `threading` and any call entering the phpy/TypePHP bridge from a non-main thread. |
||||
- `asyncio`, Python coroutines, `async`/`await`, and cross-language event loop scheduling. |
||||
- CPython subinterpreters and the per-interpreter GIL mode. |
||||
- Python as the entry point independently loading a TypePHP program. |
||||
- Generating a Python extension or registering TypePHP functions, classes, or objects as importable Python modules. |
||||
- Runtime reflection generating TypePHP static types. |
||||
- Automatic import of `from module import *`. |
||||
- pickle/serialization of Python objects. |
||||
- Passing `PyObject` across processes. |
||||
- Python interop in the WASM target. |
||||
|
||||
Forbidden capabilities must have explicit guardrails: the compiler gives a FatalError for `threading`, `_thread`, `asyncio`, and subinterpreter APIs that can be statically recognized; phpy records the owner thread that created the runtime and refuses to enter the ZendVM bridge from other threads. Dynamic imports, reflection, or third-party packages cannot be fully recognized by the compiler, so runtime checks cannot be omitted. |
||||
|
||||
Computation threads inside third-party native packages that are completely closed and never enter the CPython API or the phpy/ZendVM bridge do not belong to the Python thread capability here; they are invisible to TypePHP and must not produce cross-thread callbacks. |
||||
|
||||
## 21. TDD and Test Gates |
||||
|
||||
The implementation and refactoring of this project must strictly follow TDD; the order must not be reversed: |
||||
|
||||
1. Write tests based on the confirmed design semantics. |
||||
2. Run the tests and confirm that they fail because the target capability is not yet implemented or an existing bug exists. |
||||
3. Write the minimal implementation that makes the tests pass. |
||||
4. Run the relevant tests and the full regression. |
||||
5. Refactor, clean up, and optimize under test protection. |
||||
6. Run the full regression, memory checks, and coverage checks again. |
||||
|
||||
It is forbidden to first complete the implementation and then add tests that only verify the current implementation details. Every bug must first have a regression test that reliably reproduces the problem. |
||||
|
||||
### 21.1 Three-Layer Mandatory Testing |
||||
|
||||
#### PHPUnit |
||||
|
||||
The TypePHP repository's PHPUnit is used to verify the compiler itself: |
||||
|
||||
- Python import and special name resolution. |
||||
- AST, symbol table, and type inference. |
||||
- C++ code generation. |
||||
- Compile-time errors and diagnostic locations. |
||||
- Compile-time diagnostics for permanently disabled capabilities, and successful code generation in environments without phpy. |
||||
- Boundary logic that does not require starting CPython. |
||||
|
||||
The phpy repository's existing PHPUnit is used to verify the ZendVM/PHP Facade and the shared Runtime: |
||||
|
||||
- Public PHP APIs such as `PyCore`, `PyObject`, and `PyDict`. |
||||
- Conversion between PHP values and Python objects. |
||||
- Mapping of Python exceptions to `PyError`. |
||||
- That the opcode handler and the Zend dynamic-call API used by TypePHP have consistent semantics. |
||||
- The object-retention call paths that TypePHP needs. |
||||
- GIL, reference counting, destruction, and exception paths. |
||||
|
||||
#### PHPT |
||||
|
||||
Used to verify end-to-end behavior of the language and runtime from the TypePHP user's perspective: |
||||
|
||||
- Import, `module\name` package variable reads, `module\name()` callable calls, and keyword arguments. |
||||
- That `use python\module as alias` and hand-written `$alias = PyCore::import('module')` are equivalent in result, exception, and object identity. |
||||
- Multiple aliases, nested modules, and repeated imports across `.cc` files. |
||||
- When there is only `use python\module` without accessing any related symbol, no helper is generated, no import is called, and the existence of that Python module is not checked. |
||||
- The same fully qualified module name is allocated only one ID across functions and across `.cc` files, and the import API is only called on first access. |
||||
- On import failure, the map slot stays `UNDEF`; after the exception is caught, the next access can retry. |
||||
- Request clean performs `zval_ptr_dtor()` on each module zval and restores it to `UNDEF`; valid Zend objects must not be directly `memset`. |
||||
- Deleting or replacing `sys.modules` entries does not change already-cached TypePHP module bindings. |
||||
- Compile-time FatalError for the legacy `module::name` / `module::$name` class member syntax, and runtime exceptions for nonexistent members and non-callable members. |
||||
- Attributes, subscripts, iteration, operators, and truthiness. |
||||
- Conversion from TypePHP arguments to Python, and explicit conversion of Python return values. |
||||
- Empty TypePHP arrays default to a Python list, and array recursive deep copying, exception abort, and repeated conversion behavior. |
||||
- Python builtins, module functions, methods, and operation results do not implicitly become TypePHP scalars. |
||||
- Explicit boundaries such as `$obj->toValue()->toInt()`, `$obj->toArray()`, `python\scalar($obj)->toInt()`, and the subsequent ordinary TypePHP conversions that restore static types and operation rules. |
||||
- Conversion from Python exceptions to TypePHP exceptions. |
||||
- When phpy is not loaded, the first Python call throws a PHP `Error`, while a merely declared unused Python `use` does not error. |
||||
- TypePHP callables being called back by Python. |
||||
- Reference counting, object destruction, and repeated calls. |
||||
- The real output of compiled programs, not just checking the generated code string. |
||||
|
||||
#### pytest |
||||
|
||||
pytest is used for regression testing of phpy's own existing Python-facing bridge; it does not mean TypePHP will generate a Python extension. It needs to verify: |
||||
|
||||
- Python calling PHP functions, objects, and callables. |
||||
- Synchronous reentrancy and the phpy module lifecycle. |
||||
- Python's holding, release, and exception mapping of Zend callable/object proxies. |
||||
- The guard that permanently forbids entering the ZendVM from Python threads. |
||||
|
||||
The three layers of testing cannot replace each other. C++/GoogleTest can cover phpy's internal reference counting, RAII, and low-level conversion, but cannot replace PHPUnit, PHPT, or pytest. |
||||
|
||||
### 21.2 Test Matrix for Each Semantic |
||||
|
||||
Each supported capability should at least consider the following dimensions: |
||||
|
||||
- The normal path. |
||||
- Error types and error messages. |
||||
- Boundary values and empty values. |
||||
- Python subclasses and dynamic protocols. |
||||
- TypePHP → Python → TypePHP reentrancy. |
||||
- Python → TypePHP → Python synchronous reentrancy initiated only by TypePHP and occurring through callable proxies. |
||||
- Normal destruction and exceptional destruction. |
||||
- Repeated execution, `sys.modules` identity, and repeated import not re-executing module code. |
||||
- Debug, Release, and supported platforms. |
||||
|
||||
Conversion tests must include: |
||||
|
||||
- `PHP_INT_MIN/PHP_INT_MAX` and out-of-range Python ints. |
||||
- `NaN`, `INF`, `-INF`, and negative zero. |
||||
- Empty strings, Unicode, invalid UTF-8, and bytes containing NUL. |
||||
- Empty list/dict, mixed keys, deep containers, recursive containers, and cyclic references. |
||||
- Identity of the same Python object after multiple wrappings. |
||||
|
||||
### 21.3 Memory and Stability Tests |
||||
|
||||
Modifications involving `PyObject*` or `zval` ownership must, in addition to functional tests, also perform: |
||||
|
||||
- PHP memory leak report. |
||||
- Python debug build/refcount checks (when the environment is available). |
||||
- ASan/UBSan builds. |
||||
- Exception injection tests covering every branch that can return early. |
||||
- Stress tests that cyclically create and destroy objects. |
||||
- Tests where cross-VM proxy objects still exist at process exit. |
||||
|
||||
Coredumps, leaks, or uncleaned pending exceptions must not be marked as "expected behavior" to bypass tests. |
||||
|
||||
### 21.4 Coverage Requirements |
||||
|
||||
- Every normative behavior in the design document must correspond to at least one test. |
||||
- New and modified bridge code needs to cover both normal and error branches. |
||||
- Overall project coverage must not decrease because of this feature. |
||||
- For GIL, reference counting, exception, and destruction code, line coverage alone is not sufficient; the branch matrix must be manually inspected. |
||||
- The final coding plan must first list the test checklist, then the implementation tasks. |
||||
|
||||
## 22. Confirmed and Pending Questions |
||||
|
||||
Confirmed: |
||||
|
||||
1. Python interop is an optional extension-level feature; TypePHP does not link or compile-time check `libphpy.so`, and if phpy is not loaded at the first actual call, Zend throws a PHP `Error`. |
||||
2. TypePHP adopts explicit conversion as much as possible and does not inherit all of phpy's implicit conversion behavior. |
||||
3. TypePHP operators are rewritten at compile time into Python standard library calls of the form `operator\add($left, $right)`, without using phpy opcode handlers and without generating phpy C++ symbol calls. |
||||
4. The `python` root namespace is case-insensitive; all Python symbols after it are case-sensitive. |
||||
5. `python` is a special language namespace handled by the compiler. |
||||
6. Runtime classes continue to use phpy's public names such as `PyObject` and `PyDict`. |
||||
7. Construction syntax such as `python\dict()` is syntactic sugar for existing phpy class constructors; `python\print()` and the like are syntactic sugar for `PyCore` APIs. |
||||
8. `new PyList()` and `python\list()` have the same `PyList` typed object type and optimization capability. |
||||
9. phpy solves runtime concerns; TypePHP only dynamically calls the phpy Facade through cached `zend_function*` and the PHPX/Zend generic object API. |
||||
10. TypePHP's Python-specific implementation and tests are placed in independent subdirectories and integrated into the general compilation flow through controlled entry points. |
||||
11. Python threads, `asyncio`, and subinterpreters are permanently forbidden and are not future compatibility targets. |
||||
12. `===` / `!==` map to Python identity `is` / `is not` respectively; `==` / `!=` use Python value comparison. |
||||
13. Only TypePHP actively calling Python is supported; no Python extension is generated, no `#[PythonExport]` is provided, and no TypePHP symbols are registered with Python. |
||||
14. The CPython and bridge lifecycle fully reuses phpy's `MINIT/RINIT/RSHUTDOWN/MSHUTDOWN` entry points. |
||||
15. Python package variables are read using the PHP namespace constant syntax `math\pi`, but at runtime a dynamic Python attribute lookup is performed; `math::pi` and `math::$pi` are incorrect class member expressions. |
||||
16. `np\array()` means reading and calling a Python package member; that member can be a function, a class, or another callable, and the specific type is determined by the Python runtime. |
||||
17. `PyObject` can be mixed with TypePHP values in operations; after TypePHP operands are converted to Python objects, the entire operation is executed by the CPython protocol, and the result remains a `PyObject`. |
||||
18. The results of Python functions, methods, class construction, and builtin calls all remain `PyObject` or known phpy subclasses; phpy return value implicit conversion is disabled. |
||||
19. `PyObject::toValue()` is an explicit scalar/container conversion method, equivalent to `python\scalar()`; `PyObject::toArray()` only accepts convertible containers and iterators, and unsupported types return an empty array. After conversion, ordinary TypePHP conversion can continue to be used, for example `$obj->toValue()->toInt()`. |
||||
20. When TypePHP calls Python, all arguments are automatically converted to Python types; TypePHP arrays are recursively deep-copied, and empty arrays default to a Python list. |
||||
21. Performance-sensitive code should reuse proxy objects such as `PyDict`, `PyList`, and `PyStr`, avoiding repeated conversion and deep copying of the same TypePHP value. |
||||
22. TypePHP's main language increment is `use python\...` and module aliases; when using aliases, phpy import is called through a lazy indexed map of the same kind as `funcMap`, and other runtime capabilities preferentially reuse phpy directly. |
||||
23. `use python\module` is handled entirely by PHP namespace resolution; when the current `.php` file does not actually access symbols that resolve to that module, no helper is generated and no runtime import is performed. |
||||
24. An ID is only allocated for a fully qualified module name when `module\attr` or `module\func()` is found; unused `use` does not occupy a map slot and does not perform import. |
||||
25. `pythonModuleMap`, like `funcMap`, is declared centrally and looked up lazily by ID; the difference is that modules are stored as owning-reference Zend object zvals and must be `zval_ptr_dtor()`-ed one by one during request clean and restored to `UNDEF`. |
||||
26. `sys.modules` is responsible for global load state and identity; `pythonModuleMap` only represents the module bindings that TypePHP has completed and cached. |
||||
27. TypePHP generated code only depends on PHPX/ZendVM; `PyCore::import()`, builtins, object methods, and conversions are all resolved as `zend_function*` dynamic calls. |
||||
28. Python operators implicitly use the `python\operator` module; the complete operator protocol is handled by CPython `operator` functions, without directly calling dunders and without TypePHP implementing reflected fallback. |
||||
|
||||
Module namespace attributes are read-only in the first version. A PHP namespace constant expression cannot be an assignment target; if write capability is added later, an explicit API should be used, and tests should be added first after the semantics are determined. |
||||
@ -0,0 +1,103 @@ |
||||
# TypePHP Python Interop Phased Implementation Plan |
||||
|
||||
> This plan takes `python/design.md` as its specification. Each phase strictly follows: first add PHPUnit/PHPT/pytest tests and confirm they fail, then implement, then run the relevant tests and the full regression. |
||||
|
||||
## Phase 1: Python module names, use, and lazy binding |
||||
|
||||
The goal is to complete a minimal runnable closed loop, without implementing operators and general conversion: |
||||
|
||||
1. Recognize `python\module\member()` / `python\module\member` in the global namespace, `\python\module\member()` / `\python\module\member` in other namespaces, the optional `use python\module` shorthand, case-insensitive root names, and case-sensitive Python subsequent names. |
||||
2. Do not build a Python-specific alias table, and do not specially handle `use`; use PHP's ordinary namespace, `use function`, `use const`, `as` alias, conflict checking, and fully qualified name resolution. Inside a namespace it must be written as `\python\module`; the relative `python\module` still resolves to a name under the current PHP namespace. |
||||
3. Allocate a module ID only when `python\module\attr`, `python\module\func()`, or their alias forms appear. |
||||
4. Generate a `pythonModuleMap` of the same kind as `funcMap`, a lazy getter, and request-clean code. |
||||
5. Dynamically call `PyCore::import()` using the Zend class/function map; do not include, link, or detect phpy. |
||||
6. Use the Zend object API to read module attributes and call module callables. |
||||
7. When phpy is not loaded, throw a PHP `Error` at the first actual use; an unused Python use does not trigger an error. |
||||
|
||||
Test order: PHPUnit code generation and diagnostic tests → PHPT runtime tests → existing compiler regression. |
||||
|
||||
Implementation status: completed. Fully qualified names and any alias share the same runtime slot per Python dotted module name; fully qualified names do not require `use`, and both syntaxes lazy-import on first actual execution. Before recognition, PHP namespace resolution is strictly applied: for example, in `namespace App`, the relative name `python\math\sqrt()` is an ordinary `App\python\math\sqrt()`, and only `\python\math\sqrt()` points to the Python root namespace. |
||||
|
||||
## Phase 2: builtins, construction syntactic sugar, and static types |
||||
|
||||
1. `python\name()` is dynamically called through the phpy Zend Facade: explicit `PyCore` methods are reused directly, and other names go through a Python `builtins` module lookup. |
||||
2. `python\list/dict/tuple/set/str/object()` maps to existing phpy Zend classes or methods. |
||||
3. `new PyList()` and `python\list()` and the like obtain the same logical static type. |
||||
4. Python call results remain `PyObject` or known phpy subclasses, turning off implicit scalar conversion on the TypePHP path. |
||||
5. Tests for missing phpy, nonexistent builtins, argument errors, and exception mapping. |
||||
|
||||
Implementation status: completed. TypePHP lazily enables phpy's `return_as_object` the first time a Python expression is actually executed; merely declaring unused Python symbols still does not trigger a runtime dependency. |
||||
|
||||
## Phase 3: argument conversion and explicit result conversion |
||||
|
||||
1. TypePHP arguments are automatically converted to Python values after left-to-right evaluation. |
||||
2. Conversion of scalars, arrays, empty arrays, nested containers, and TypePHP callables. |
||||
3. Leave the Python object rules via `$py->toValue()` or `python\scalar($py)`; when a native type is needed, continue to use ordinary TypePHP conversion, e.g. `$py->toValue()->toInt()`. Containers and iterators can directly use `$py->toArray()`. |
||||
4. Tests for deep copying, recursive containers, overflow, Unicode/bytes, and exception paths. |
||||
5. Review and refactor phpy's conversion strategy, removing global temporary conversion state that affects synchronous reentrancy. |
||||
|
||||
Implementation status: the core boundary is complete. TypePHP arguments are strictly evaluated left to right, supporting scalars, empty arrays, nested list/dict, and callables; `PyObject::toValue()` and `python\scalar()` reuse phpy's explicit conversion entry point, and `PyObject::toArray()` converts supported containers and iterators. phpy has removed the process-level conversion function pointer, replacing it with a local stateful converter, RAII recursion protection, and a 128-level depth limit, and covers invalid UTF-8, PHP self-referential arrays, and Python cyclic container error paths. The final language mapping for Python big integers and bytes remains in this phase's follow-up work. |
||||
|
||||
## Phase 4: operators |
||||
|
||||
1. Rewrite operators into dynamic calls to the Python standard library `operator` module. |
||||
2. Mixed operands are first converted to `PyObject`. |
||||
3. Strictly guarantee left-to-right evaluation, each evaluated once. |
||||
4. Use `operator.is_/is_not/truth` to implement identity and truthiness, and `iadd/isub/...` to implement compound assignment. |
||||
5. Verify that `operator` automatically handles `NotImplemented`, reflected dunders, and subclass priority. |
||||
6. Against phpy's opcode-handler behavior, fix existing issues such as `/` incorrectly mapping to floor division. |
||||
|
||||
Implementation status: completed. Binary arithmetic, bitwise operations, comparison, `===`/`!==`, unary operations, conditional truthiness, short-circuit logic, and compound assignment are all executed through the implicit `operator` module binding; `/` uses `truediv`. Mixed TypePHP operands are converted by phpy at the call boundary, results continue to remain `PyObject`, and comparison and truthiness results explicitly converge to a TypePHP `bool`. Attribute and subscript lvalues are written back through the dynamic write protocol of Phase 5. |
||||
|
||||
## Phase 5: the complete object protocol |
||||
|
||||
1. Reading, writing, and deleting Python object attributes. |
||||
2. Subscript read/write, delete, and `isset()`. |
||||
3. iterator/foreach. |
||||
4. Synchronous reentrancy of Python callables and TypePHP callable proxies. |
||||
5. keyword arguments, argument unpacking, and error semantics. |
||||
|
||||
Implementation status: completed. Dynamic attributes, unknown methods, subscripts, deletion, `isset()`, `foreach`, and callables of Python proxies all reuse phpy's Zend object protocol, without generating phpy C++ symbols. Method, attribute, subscript, and callable results continue to propagate as `PyObject`, supporting chained access and subsequent Python operations. Named arguments and unpacking reuse the unified call argument pipeline and keep left-to-right evaluation; attribute and subscript compound assignments use the returned object of `operator.i*()` to write back the original lvalue. |
||||
|
||||
phpy simultaneously completed object protocol hardening: `__set()` conversion reference release, `__unset()`, list/tuple negative indexing, list deletion, `isset()` semantics for missing keys and Python `None`, deletion and contains state checks, and iterator/count exception propagation. The related bugs are independently covered by phpy PHPUnit and TypePHP PHPT. |
||||
|
||||
## Phase 6: phpy stability and performance finishing |
||||
|
||||
1. Full audit of the CPython/ZendVM lifecycle, GIL, and owned/borrowed/stolen references. |
||||
2. Audit of Python/Zend exception state and traceback. |
||||
3. Cross-VM reference cycles, destruction, and exception injection tests. |
||||
4. ASan/UBSan, PHP leak report, Python debug build, and stress tests. |
||||
5. Benchmark dynamic Zend calls, the module map, argument conversion, and `operator` module calls; only optimize hotspots proven by data. |
||||
6. Full PHPUnit, pytest, PHPT, and existing TypePHP compiler regression. |
||||
|
||||
Implementation status: completed. The first round of CPython failure-path audits covered the construction and subscript writes of general objects, list, dict, tuple, and set, as well as sequence/set `contains()`. PHP-to-Python key/value conversion failures are now immediately mapped to `PyError`, all newly acquired references are released by scope guards; construction failure no longer leaves an unhandled CPython error indicator, and the `-1` error result of `contains()` is no longer misjudged as `true`. Paths such as invalid UTF-8, unhashable set members, and containers remaining usable after failure have phpy PHPUnit regression tests. |
||||
|
||||
The second round covered module import, exception conversion, callable checks, and the explicit iterator API. New references transferred to the caller by `PyImport_ImportModule()`, `PyErr_Fetch()`, and `PyIter_Next()` are now uniformly released after the Zend wrapper acquires an independent reference; repeated imports, Python exceptions, or explicit iterator next no longer continuously increase the reference count. Calling a non-callable Python attribute or `PyObject` reliably throws `PyError(TypeError)`, no longer silently returning `null` because `PyCallable_Check()` did not set an error indicator. `PyCore::next()` also distinguishes normal end-of-iteration from iterator exceptions. All the above paths first establish failing phpy PHPUnit regression tests, with object call behavior additionally covered by TypePHP PHPT integration. |
||||
|
||||
The third round covered conversion failures and function caching of the `PyCore` Facade. `PyCore::eval()` throws a `PyError` immediately after globals conversion fails, and `PyCore::bytes()` uses the converted `zend_string` for non-string scalars; neither dereferences a null pointer or the wrong zval union field, which would crash the process. `PyCore::next()` simultaneously releases the iterator reference produced by argument conversion. The builtin/operator function cache uses a `std::string` content key, no longer using request-level `char*` addresses as long-term keys, and also avoids the same-named dynamic call repeatedly caching and increasing Python function reference counts; calling a builtin that exists but is not callable releases the temporary reference and throws `PyError(TypeError)`. All issues are covered by independent PHPUnit tests that first fail, of which two crashes were confirmed in isolated processes with core dumps disabled (exit code 139) before being fixed. |
||||
|
||||
The fourth round covered the Python-to-PHP synchronous callback boundary. phpy converts Python keyword arguments into Zend named parameters and stops immediately after any positional or named argument conversion fails, so a PHP callable that only received part of its arguments is never executed. PHPX generates and manages Zend `arg_info` parameter name metadata for AOT native closures, so Python kwargs can be bound to TypePHP closures by name rather than depending on parameter positions or degrading to a string callable. phpy PHPUnit, PHPX unit tests, and TypePHP PHPT respectively cover conversion failure, Zend named binding, and the complete Python→TypePHP callback chain. |
||||
|
||||
The fifth round covered exceptions and ownership when Python strings cross the Zend boundary. When a Python lone surrogate character cannot be encoded as UTF-8, `phpy.String`, dynamic PHP class names, dictionary keys, `PyObject::__toString()`, and Python exception message formatting no longer use null pointers or uninitialized lengths; before the fix, the relevant isolation tests would exit with 139 or attempt to allocate abnormally large memory. `StrObject` now has an explicit valid state, and all callers must check the conversion result before accessing the pointer; stringification of exception messages is only best-effort auxiliary information, preserving the original Python error/type/value on failure and cleaning up the temporary CPython error indicator. `new_string()` also completes Zend carrier destruction registration, and fixed-length strings directly obtain a unique `zend_string` reference, eliminating the leak and uninitialized zval on the success path. |
||||
|
||||
The sixth reference audit fixed the new-reference leak in `PySequence::slice()`. After the slice is wrapped as a Zend `PyObject`, the original ownership returned by the CPython API is released while the wrapper keeps its own reference; a `sys.getrefcount()` stress test verifies that repeatedly creating and destroying slices does not continue to increase the element reference count. Slice creation failure is also converted to `PyError` before touching a null pointer. |
||||
|
||||
The seventh round covered the Python operator protocol of dynamic PHP. The phpy opcode handler now uses CPython `PyNumber_*` / `PyObject_RichCompareBool()`, `/` and `/=` use true division, compound assignment updates the Zend lvalue with the object returned by the in-place API, and correctly handles immutable Python objects, Zend reference variables, expression results, and the lvalue state after failure. `===` / `!==` use Python object identity, bool cast, `!`, and conditional branches use the Python truth protocol; new references on the PHP operand conversion, result, and exception paths are uniformly managed by RAII guards. TypePHP's `operators.phpt` runs with both ZendPHP + opcode handler and AOT + `operator` module, so it is the output-consistency gate for the two implementations. |
||||
|
||||
Unary positive/negative operations in dynamic code have been confirmed as a compatibility boundary: PHP compiles `-$value` / `+$value` into multiplication by `-1` / `1`, and the opcode handler cannot distinguish this from explicit multiplication in the source. Dynamic ZendVM code no longer attempts to rewrite the AST, but explicitly retains the `$value * -1` / `$value * 1` behavior. Python builtin numerics and common objects such as NumPy usually produce consistent results, but a custom object's `__neg__()` / `__pos__()` may differ from its `__mul__()`. AOT TypePHP retains the original AST and still lowers to `operator.neg()` / `operator.pos()` respectively, so the semantics are unaffected. The external user document `python.md` has listed this difference as a compatibility limitation. |
||||
|
||||
The performance benchmarks separately cover module properties, operator module calls, existing `PyObject` arguments, and PHP scalar argument conversion. In the current unoptimized build, module properties are about 0.7–0.9 μs/op, and operator calls about 1.8–2.5 μs/op; whether arguments are pre-wrapped as `PyObject` shows no stable difference. After attempting to eliminate the per-access reference count of the module map with an indirect zval wrapper, the A/B medians remained within the same noise range, so the more lifetime-sensitive optimization with unproven benefit was not retained. |
||||
|
||||
phpy's CMake Python-extension target also completed the out-of-tree build fix, and is compatible with the `int` ABI of older PHP and the `zend_result` ABI of newer PHP by deriving the return type from Zend's standard cast handler. Both PHP 8.1 and PHP 8.4 have completed clean-build verification. |
||||
|
||||
The memory gate uses Valgrind Memcheck. Tests disable the Zend allocator and PCRE JIT, and in a minimal independent process loop 100 times each over PHP Closure kwargs callbacks, callable PHP object kwargs callbacks, sequence slice creation/destruction, object stringification of invalid Unicode, dictionary key conversion, and exception formatting. The result is 0 invalid-access, 0 definite leak, 0 indirect leak; the 493,106 bytes retained by PHP/CPython at process exit are all still-reachable and not counted as leaks. The ASan extension cannot be safely `dlopen`-ed into the current non-ASan PHP with `RTLD_DEEPBIND` enabled, so this round uses Valgrind, which does not require recompiling PHP, as the memory checking tool. |
||||
|
||||
Phase 6 final gate results: phpy PHPUnit 135 tests / 469 assertions passed (1 existing warning, 1 environment-related skip), pytest 26/26 passed, TypePHP Python PHPT 14/14 passed, TypePHP PHPUnit 1103 tests / 2729 assertions passed; the TypePHP compiler full PHPT has 934 items in total, of which 932 PASS, 2 SKIP, 0 FAIL, 0 WARN. The dynamic operator stress test under Valgrind is 0 invalid access, 0 definite leak, 0 indirect leak. |
||||
|
||||
## Phase Gates |
||||
|
||||
- Do not start implementation before the current phase's failing tests are established. |
||||
- Do not enter the next phase before all tests of the current phase pass. |
||||
- phpy behavior changes must first add PHPUnit/pytest tests in the phpy repository. |
||||
- Every fixed bug must retain an independent regression test. |
||||
- Do not mask implementation differences by modifying third-party test expectations. |
||||
@ -0,0 +1,98 @@ |
||||
# Python Tools Submodule |
||||
|
||||
TypePHP integrates the Python IDE helper generator and the Python source converter into `tpc`. Both are located in an independent |
||||
`src/PythonTools` directory, only reusing the `tpc` command entry point, and do not enter the normal PHP preprocessing, C++ generation, and compilation pipeline. |
||||
|
||||
## Python namespace IDE helper |
||||
|
||||
```shell |
||||
./tpc --gen-python-helper math |
||||
./tpc --gen-python-helper numpy.linalg |
||||
./tpc --gen-python-helper numpy --output-dir .ide-helper |
||||
``` |
||||
|
||||
The command imports the specified Python module through PHPy, and uses the Python `inspect` API to collect functions, parameters, classes, methods, and |
||||
module attributes. The PHPy extension and the target Python module must be installed in the host environment where `tpc` is executed. |
||||
|
||||
By default, generated files are located in `ide-helper` in the current directory. `--output-dir` can replace this output root directory, supporting both |
||||
paths relative to the current directory and absolute paths: |
||||
|
||||
```text |
||||
ide-helper/python/math.php |
||||
ide-helper/python/numpy/linalg.php |
||||
ide-helper/python.php |
||||
ide-helper/PyObject.php |
||||
``` |
||||
|
||||
Each time a module helper is generated, the Python `builtins` are also scanned and the root namespace file |
||||
`python.php` is generated, providing IDE completion for builtin symbols such as `python\tuple()` and `python\len()`. This file is |
||||
regenerated according to the current Python environment. |
||||
|
||||
When a module helper is first generated, the common `PyObject.php` is also generated. It contains method hints for dynamic access, calls, |
||||
array access, iteration, and `toArray()`, `toValue()` of `PyObject`, shared by all Python module helpers. If this file |
||||
already exists, the generator keeps the original file and does not overwrite it. |
||||
|
||||
Generated content uses TypePHP's module-as-namespace form, e.g. `python\math\sqrt()`, and is compatible with the IDE name resolution of ordinary |
||||
`use`, `use function`, and `use const`. The end of the file contains `die`, which is used to explicitly |
||||
terminate the program when executed by mistake. Helpers can only be handed to the IDE for indexing; they cannot be included, nor added to a TypePHP project's sources or compilation inputs. |
||||
|
||||
`PyObject::IDE_HELPER_ONLY` is a hint constant shared by all helpers. The method bodies of non-`void` stubs use |
||||
`die(\PyObject::IDE_HELPER_ONLY)` to satisfy the IDE's control-flow checks on return types, no longer producing a "missing return |
||||
statement" diagnostic. Module attributes use namespace `const` declarations, supporting the IDE's constant completion and `use const`. |
||||
PHP 8.1 and above allow `new` in constant initializer expressions. Module attributes therefore directly use an IDE-analysis-only |
||||
`PyObject` instance as a placeholder value: |
||||
|
||||
```php |
||||
const pi = new \PyObject(); |
||||
``` |
||||
|
||||
This way the IDE precisely recognizes the constant as `PyObject`, instead of inferring a wrong type from `null`. |
||||
|
||||
The common `PyObject` helper also declares TypePHP's virtual keyword methods, including `toInt()`, `toFloat()`, |
||||
`toString()`, `toBool()`, `toStream()`, high-precision type conversions, `toObject()`, `toAny()`, and `toRef()`. |
||||
These declarations are only used for IDE completion; calls are expanded at compile time and are not entity methods of the PHPy `PyObject` runtime class. |
||||
`toArray()` and `toValue()` are still real methods provided by PHPy. |
||||
|
||||
Python class constructors explicitly call `parent::__construct()`. If a Python object defines `count()`, the helper |
||||
does not declare it again, because `PyObject::count(): int` is already used for PHP `Countable`. When the Python object's own |
||||
`count()` needs to be called, it should be written explicitly as `$object->__call('count', $arguments)`. |
||||
|
||||
PHP function/class names are case-insensitive, while Python names are case-sensitive; PHP reserved words also cannot be declared as ordinary |
||||
stub symbols. The generator reports symbols that cannot be expressed with legal PHP declarations as comments, and does not rename the Python API on its own. |
||||
The call syntax of `python\print()` is legal, but PHP forbids declaring a function named `print`, so a plain |
||||
PHP helper file cannot provide a syntax-error-free symbol declaration for it. Reserved words such as `list`, `int`, and `float` have |
||||
the same limitation. |
||||
|
||||
## Python to TypePHP |
||||
|
||||
```shell |
||||
./tpc --convert-python-to-php script.py > script.php |
||||
``` |
||||
|
||||
The converter invokes `python3` in the PATH to parse the Python AST, then outputs PHP source using the TypePHP Python namespace |
||||
syntax. Ordinary module imports are converted to namespace imports: |
||||
|
||||
```python |
||||
import math |
||||
print(math.sqrt(16)) |
||||
``` |
||||
|
||||
```php |
||||
use python\math; |
||||
|
||||
function main(): void |
||||
{ |
||||
python\print(math\sqrt(16)); |
||||
} |
||||
``` |
||||
|
||||
Currently it supports ordinary imports, functions, assignments, calls, container literals, basic operations, single comparisons, if/while/for, |
||||
lambda, and basic f-strings. Module top-level variables are converted to PHP globals to preserve the ability of functions to read module variables. |
||||
When semantics can be strictly preserved, the converter directly uses PHP native syntax: `print()` with no arguments or safely convertible arguments |
||||
generates `echo` with a newline, and `sys.exit()` with integer literal exit codes generates `exit`. `print()` with |
||||
`sep`, `end`, `file`, or `flush` arguments, and `sys.exit()` with strings or objects, |
||||
are not fully consistent with PHP behavior and remain as Python calls. |
||||
|
||||
The converter follows the principle of "reject when semantics cannot be reliably preserved". Not-yet-completed syntax such as class, async, generator, try/with, decorator, |
||||
destructuring assignment, chained comparison, nested functions, and loop-else throws an error with the |
||||
source file and line number, rather than generating PHP code that looks usable but is semantically wrong. |
||||
@ -0,0 +1,22 @@ |
||||
## Compilation |
||||
|
||||
```shell |
||||
./tpc --wasm test.php |
||||
``` |
||||
|
||||
After a successful compilation, by default only the WASI 0.2 Component `test.wasm` executable by Wasmtime is generated. WASI 0.1 is not supported. |
||||
The generated C++ source is written to `build/` by default, and can also be specified via `--build-dir <directory>`. |
||||
|
||||
## Execution |
||||
|
||||
```shell |
||||
wasmtime test.wasm |
||||
``` |
||||
|
||||
## Chrome |
||||
|
||||
```shell |
||||
./tpc --wasm=browser test.php |
||||
``` |
||||
|
||||
Browser mode additionally generates the `test.browser/` Jco module and requires `jco` to be on the `PATH`. The full browser demo is located in the repository's `examples/wasm-hello/`, and is built using the `project.yml` with `wasm: browser`. TypePHP executes in a dedicated Worker; the default filesystem resides in memory, and OPFS snapshot persistence can be explicitly enabled. Network sockets, processes, shell, and signals are explicitly not supported under the WASI target. |
||||
Loading…
Reference in new issue