Compare commits
267 Commits
speed_buil
...
master
| Author | SHA1 | Date |
|---|---|---|
|
|
b906bfce1a | 12 hours ago |
|
|
d0ff8c5e4f | 12 hours ago |
|
|
320ab547a7 | 13 hours ago |
|
|
057c217800 | 13 hours ago |
|
|
3452ddfd6e | 13 hours ago |
|
|
d86d3f431d | 13 hours ago |
|
|
bf0d7aa3a4 | 13 hours ago |
|
|
1c300bf35a | 13 hours ago |
|
|
287a1f5d84 | 13 hours ago |
|
|
7862541161 | 13 hours ago |
|
|
a0c873261c | 13 hours ago |
|
|
250a384bb8 | 14 hours ago |
|
|
dfeb71dbb5 | 14 hours ago |
|
|
2d81626a84 | 14 hours ago |
|
|
589f1494d4 | 14 hours ago |
|
|
c4ce700868 | 19 hours ago |
|
|
97069e5d5b | 2 days ago |
|
|
96ab6a53ef | 2 days ago |
|
|
e019f21550 | 2 days ago |
|
|
b493ac79c5 | 2 days ago |
|
|
9e0984acee | 2 days ago |
|
|
89f3cb485b | 2 days ago |
|
|
7159cf9b78 | 2 days ago |
|
|
ee2afb08f7 | 2 days ago |
|
|
45f6b8a163 | 2 days ago |
|
|
22486ec56d | 2 days ago |
|
|
9fd46c22e3 | 2 days ago |
|
|
c8d22b66fd | 2 days ago |
|
|
d60fa56470 | 2 days ago |
|
|
a44fd22e50 | 2 days ago |
|
|
560fa44160 | 2 days ago |
|
|
0e6d65c6e1 | 2 days ago |
|
|
cfd8e7ebbe | 2 days ago |
|
|
d11637faf0 | 2 days ago |
|
|
f2db98b7da | 2 days ago |
|
|
2ef53e7eeb | 2 days ago |
|
|
7b05882c10 | 2 days ago |
|
|
4438c5807d | 2 days ago |
|
|
0f430e85a6 | 2 days ago |
|
|
61d53179d1 | 2 days ago |
|
|
974d65a0d1 | 2 days ago |
|
|
fea85e5fcb | 2 days ago |
|
|
7dd0e703b8 | 2 days ago |
|
|
608c5e13e5 | 2 days ago |
|
|
f835f26252 | 2 days ago |
|
|
e4a863c19e | 2 days ago |
|
|
7d7977fc5a | 2 days ago |
|
|
6df71ec557 | 2 days ago |
|
|
d698f4110c | 2 days ago |
|
|
e8927bade0 | 2 days ago |
|
|
f6394934c5 | 2 days ago |
|
|
8b1c8e7d26 | 2 days ago |
|
|
a4dee96fcf | 2 days ago |
|
|
0821f62f20 | 2 days ago |
|
|
7fa95d449f | 2 days ago |
|
|
e0c1298798 | 3 days ago |
|
|
d480c5e7cb | 3 days ago |
|
|
e44007846b | 3 days ago |
|
|
91f5daf457 | 3 days ago |
|
|
9b02dcd693 | 3 days ago |
|
|
c8148018f8 | 3 days ago |
|
|
3189d5bd7f | 3 days ago |
|
|
7f0a8181f5 | 3 days ago |
|
|
fb79fb832c | 3 days ago |
|
|
019c626c37 | 3 days ago |
|
|
d8b9dc1a00 | 3 days ago |
|
|
a8f7c29ea0 | 3 days ago |
|
|
a70a0ad078 | 3 days ago |
|
|
06023674fe | 3 days ago |
|
|
d1d362d779 | 3 days ago |
|
|
eccf5a0ef1 | 3 days ago |
|
|
1d1a50c3d3 | 3 days ago |
|
|
d19809af9b | 3 days ago |
|
|
1488563a27 | 3 days ago |
|
|
c9d9f58de0 | 3 days ago |
|
|
8918fae3ba | 3 days ago |
|
|
ec251655ba | 3 days ago |
|
|
0f335a8135 | 3 days ago |
|
|
d321666721 | 3 days ago |
|
|
98321138ce | 3 days ago |
|
|
f2b0e9add9 | 3 days ago |
|
|
4a9b918e7c | 3 days ago |
|
|
1b92ae6e7b | 3 days ago |
|
|
6e593b3d46 | 4 days ago |
|
|
af2b03c111 | 4 days ago |
|
|
8ddd04e1b1 | 4 days ago |
|
|
ee1d6e7b54 | 4 days ago |
|
|
fef462af33 | 4 days ago |
|
|
875f123403 | 4 days ago |
|
|
ed27572ca1 | 4 days ago |
|
|
3d2933ec7f | 4 days ago |
|
|
87c328b3db | 4 days ago |
|
|
063476360b | 5 days ago |
|
|
022ca55ff0 | 5 days ago |
|
|
ced2ebb1ba | 5 days ago |
|
|
7973f99f47 | 5 days ago |
|
|
74274f8328 | 5 days ago |
|
|
6693d95cd6 | 5 days ago |
|
|
8f32497de7 | 5 days ago |
|
|
1b235d0769 | 5 days ago |
|
|
8c6ea1d14e | 5 days ago |
|
|
0ac6dfe499 | 5 days ago |
|
|
a6cc11c6d7 | 5 days ago |
|
|
2d14c9e432 | 5 days ago |
|
|
7f587749f1 | 5 days ago |
|
|
821f639e8f | 5 days ago |
|
|
983333dc2e | 5 days ago |
|
|
978e553480 | 5 days ago |
|
|
e4817819dc | 5 days ago |
|
|
8a30c5e58d | 5 days ago |
|
|
0061d1595a | 5 days ago |
|
|
2c74be470e | 5 days ago |
|
|
2986f3db70 | 5 days ago |
|
|
e4bb7b2605 | 5 days ago |
|
|
305d0d09d9 | 5 days ago |
|
|
a5700d24c9 | 5 days ago |
|
|
642e9a7e4b | 5 days ago |
|
|
422bdc4ce6 | 6 days ago |
|
|
208541d00b | 6 days ago |
|
|
19a570ff3f | 6 days ago |
|
|
178636a577 | 6 days ago |
|
|
1118c1e8ec | 6 days ago |
|
|
02fc0fb319 | 6 days ago |
|
|
3def25cc3c | 6 days ago |
|
|
44095252d2 | 6 days ago |
|
|
9f1cf07511 | 6 days ago |
|
|
36fd0228f0 | 6 days ago |
|
|
4c59d9a21b | 6 days ago |
|
|
b2246cd456 | 6 days ago |
|
|
795a8e0a1d | 6 days ago |
|
|
9fc5755544 | 6 days ago |
|
|
6dcafb652b | 6 days ago |
|
|
23a2bf44d7 | 6 days ago |
|
|
6d45c64b4d | 6 days ago |
|
|
c5f478766e | 6 days ago |
|
|
f64a1d5cb7 | 7 days ago |
|
|
00420215b3 | 7 days ago |
|
|
2471ed1db7 | 7 days ago |
|
|
0a97daa663 | 1 week ago |
|
|
4e00799c4a | 1 week ago |
|
|
41e11a66e6 | 1 week ago |
|
|
0f6cc17450 | 1 week ago |
|
|
101b9f6261 | 1 week ago |
|
|
65a44c7f65 | 1 week ago |
|
|
e722501b3d | 1 week ago |
|
|
d80f4a0d20 | 1 week ago |
|
|
4713a58b06 | 1 week ago |
|
|
cb1e12f11f | 1 week ago |
|
|
7418b6247e | 1 week ago |
|
|
65d3710a61 | 1 week ago |
|
|
70d13a3601 | 1 week ago |
|
|
4b8d0eb685 | 1 week ago |
|
|
a08f73c995 | 1 week ago |
|
|
729000fa0c | 1 week ago |
|
|
0d0e62fbc7 | 1 week ago |
|
|
ab1854beb0 | 2 weeks ago |
|
|
d33073c021 | 2 weeks ago |
|
|
68bdedaa31 | 2 weeks ago |
|
|
d1704b42ef | 2 weeks ago |
|
|
f1ef66622c | 2 weeks ago |
|
|
244fa4d8b2 | 2 weeks ago |
|
|
7fece68f12 | 2 weeks ago |
|
|
464de23051 | 2 weeks ago |
|
|
2ffd4ded19 | 2 weeks ago |
|
|
2e6ddf98f1 | 2 weeks ago |
|
|
c9666664aa | 2 weeks ago |
|
|
587818d5f9 | 2 weeks ago |
|
|
7d1a90a340 | 2 weeks ago |
|
|
13344ace72 | 2 weeks ago |
|
|
7cf265cbc7 | 2 weeks ago |
|
|
de8fdaf464 | 2 weeks ago |
|
|
d5f0a113e6 | 2 weeks ago |
|
|
981b8f54d7 | 2 weeks ago |
|
|
af0b6f9582 | 2 weeks ago |
|
|
2f229ee0ac | 2 weeks ago |
|
|
ea08344c92 | 2 weeks ago |
|
|
1456305ade | 2 weeks ago |
|
|
5162261c75 | 2 weeks ago |
|
|
a536172f82 | 3 weeks ago |
|
|
aaf1ec11b3 | 3 weeks ago |
|
|
e4f6475452 | 3 weeks ago |
|
|
96960e9da1 | 3 weeks ago |
|
|
bca77588b5 | 3 weeks ago |
|
|
f79d40f8ee | 3 weeks ago |
|
|
496e6a4d69 | 3 weeks ago |
|
|
a510530d9b | 3 weeks ago |
|
|
6c9adbccad | 3 weeks ago |
|
|
343084c4ba | 3 weeks ago |
|
|
af7ae9df82 | 3 weeks ago |
|
|
42e96f1103 | 3 weeks ago |
|
|
23e88efb6a | 3 weeks ago |
|
|
463b427dc6 | 3 weeks ago |
|
|
e1018ca87e | 3 weeks ago |
|
|
bd6335ec3b | 3 weeks ago |
|
|
a79610fb2c | 3 weeks ago |
|
|
5fb3fdf490 | 3 weeks ago |
|
|
2a1ca3b1a0 | 3 weeks ago |
|
|
01ea7c40be | 3 weeks ago |
|
|
8aa8cdb065 | 3 weeks ago |
|
|
6e5fadcbc6 | 3 weeks ago |
|
|
18dd9a8a32 | 3 weeks ago |
|
|
e07b97d6cf | 3 weeks ago |
|
|
8a3b5ae9a4 | 3 weeks ago |
|
|
8e48b9d7fe | 3 weeks ago |
|
|
f5ca997e65 | 3 weeks ago |
|
|
c47ef90593 | 3 weeks ago |
|
|
b77fee697e | 3 weeks ago |
|
|
7190bea4a5 | 3 weeks ago |
|
|
6fb8187b07 | 3 weeks ago |
|
|
997bbffb65 | 3 weeks ago |
|
|
8118fbd3dc | 3 weeks ago |
|
|
e5f61eabfe | 3 weeks ago |
|
|
a3ea91918d | 3 weeks ago |
|
|
3b51eaa6d0 | 3 weeks ago |
|
|
0ba5079336 | 3 weeks ago |
|
|
7d7fa75d6f | 3 weeks ago |
|
|
64afbc9656 | 3 weeks ago |
|
|
6c97b4f301 | 3 weeks ago |
|
|
f61a542255 | 3 weeks ago |
|
|
3b6a48ef74 | 3 weeks ago |
|
|
22160661a9 | 3 weeks ago |
|
|
069d3e61bb | 3 weeks ago |
|
|
2be13eefcd | 3 weeks ago |
|
|
cb03570212 | 3 weeks ago |
|
|
70927ee68a | 3 weeks ago |
|
|
a4f1188a7e | 3 weeks ago |
|
|
1072e9d1c7 | 3 weeks ago |
|
|
1847b4c926 | 3 weeks ago |
|
|
f630fcf661 | 3 weeks ago |
|
|
525a4e4412 | 3 weeks ago |
|
|
b1b40d6a09 | 3 weeks ago |
|
|
8d4801ebd2 | 3 weeks ago |
|
|
ca41af9bb5 | 3 weeks ago |
|
|
833b74b344 | 3 weeks ago |
|
|
b69582011a | 3 weeks ago |
|
|
c4b8db030d | 3 weeks ago |
|
|
7f46445a38 | 3 weeks ago |
|
|
39354af70b | 3 weeks ago |
|
|
dbd12d249d | 3 weeks ago |
|
|
1baefe166f | 3 weeks ago |
|
|
c4404c15c5 | 3 weeks ago |
|
|
abbaa11c7c | 3 weeks ago |
|
|
3d36928e8b | 3 weeks ago |
|
|
ac16f5fbac | 3 weeks ago |
|
|
9e26887c03 | 3 weeks ago |
|
|
07c7d4b84a | 3 weeks ago |
|
|
9992793a02 | 3 weeks ago |
|
|
8503c7c32c | 3 weeks ago |
|
|
cee9ee6e1b | 3 weeks ago |
|
|
c3614aa6d3 | 3 weeks ago |
|
|
580d877ecd | 3 weeks ago |
|
|
3b3711c139 | 3 weeks ago |
|
|
b71ef5ca1a | 3 weeks ago |
|
|
9d1ba7ee7a | 3 weeks ago |
|
|
8dd33c1400 | 3 weeks ago |
|
|
db4b7f9e28 | 3 weeks ago |
|
|
2310c574ab | 3 weeks ago |
|
|
8a23368cf1 | 3 weeks ago |
|
|
ec298e66fe | 3 weeks ago |
|
|
6d7e68a9ff | 3 weeks ago |
|
|
a7b16dc1be | 3 weeks ago |
|
|
c6c04d7e96 | 4 weeks ago |
|
|
78135d75ef | 4 weeks ago |
|
|
f2c9309857 | 4 weeks ago |
|
|
09b99e357d | 4 weeks ago |
|
|
560ea46692 | 4 weeks ago |
|
|
a92fc0dfd7 | 4 weeks ago |
1046 changed files with 87444 additions and 13237 deletions
@ -1,274 +0,0 @@ |
||||
# Code Reuse Improvement Plan |
||||
|
||||
## Analysis Summary |
||||
|
||||
| Metric | Value | |
||||
|--------|-------| |
||||
| Total source lines | ~11,000 (PHP only) | |
||||
| CompilerBase | 5,917 lines, 269 methods, 20 traits | |
||||
| Gcc↔Clang duplication | ~70-80% of methods | |
||||
| Linux↔Macos duplication | ~80% of methods | |
||||
| `fatalError()` call sites | 174+ across codebase | |
||||
| Test setUp/tearDown dup | 4+ test classes | |
||||
|
||||
--- |
||||
|
||||
## Phase 1: High-Impact Backend/Platform Deduplication (P0) |
||||
|
||||
### 1.1 Extract `UnixPlatform` base class |
||||
|
||||
**Files**: `Platform/Linux.php` (249 lines), `Platform/Macos.php` (278 lines) |
||||
|
||||
These 13 methods are 100% identical between Linux and Macos: |
||||
- `getIncludeFlags()`, `getLibraryPathFlags()`, `getObjectExtension()`, `getExecutableExtension()`, `getPathSeparator()`, `getPhpDir()`, `getRpathOptions()`, `getPicFlag()`, `buildPhpIncludePaths()`, `findPhpConfig()`, `buildPhpLibPaths()` |
||||
|
||||
Near-identical with minor parameterization: |
||||
- `getLibraryFlags()` — only the regex differs (`.a|.so` vs `.a|.dylib`) |
||||
- `detectPhpLibs()` — only the lib name differs (`libphp.so` vs `libphp.dylib`) |
||||
|
||||
**Plan**: Create `UnixPlatform extends PlatformBase` between `PlatformBase` and `Linux`/`Macos`. Move all identical methods up. Add abstract `getSharedLibraryExtension()` (already exists) and a protected `getSharedLibName()` for the single differing method. |
||||
|
||||
**Expected savings**: ~180 lines removed, ~150 lines added = net ~30 lines but massive maintainability gain. |
||||
|
||||
### 1.2 Extract `GccLikeBackend` base class |
||||
|
||||
**Files**: `Backend/Gcc.php` (379 lines), `Backend/Clang.php` (515 lines) |
||||
|
||||
These methods are structurally identical with only Windows-specific branching: |
||||
- `compileFile()`, `linkObjects()`, `buildCompileCommand()`, `buildCCompileCommand()`, `buildNativeCompileCommand()`, `buildLinkCommand()`, `buildCompileOptions()`, `buildLinkOptions()`, `buildFullCompileOptions()`, `buildFullLinkOptions()` |
||||
|
||||
**Plan**: Create `GccLikeBackend extends CompilerBackend` with all shared logic. Define template-method hooks for the differences: |
||||
- `getCompilerSpecificFlags()` — empty for Gcc, MSVC compat flags for Clang/Windows |
||||
- `getOutputFlag($isWindows)` — `-o` vs `/OUT:` |
||||
- `getSanitizerFlag($type)` — handle the `address`/`addr` aliasing difference |
||||
- `getPICHandling($config)` — Gcc always adds `-fPIC`, Clang skips on Windows |
||||
|
||||
**Expected savings**: ~250+ lines removed from Gcc.php and Clang.php. Msvc.php is sufficiently different (different flag syntax) to remain standalone. |
||||
|
||||
--- |
||||
|
||||
## Phase 2: CompilerBase Internal Deduplication (P1) |
||||
|
||||
### 2.1 Consolidate Big* type dispatch in BinaryOpTrait |
||||
|
||||
**File**: `Parser/BinaryOpTrait.php` (lines 31-107) |
||||
|
||||
The three blocks for BigFloat (lines 31-54), Decimal (lines 56-76), and BigInt (lines 78-107) in `parseBinaryOp()` share identical structure: |
||||
1. Check if either operand is the big type |
||||
2. Guard against incompatible mixing |
||||
3. Convert the non-matching operand |
||||
4. Dispatch to arithmetic or comparison operator |
||||
|
||||
**Plan**: Extract `parseBigNumBinaryOp(string $type, string $left, string $right, ...)` parameterized by type name, conversion function, and operator maps. Same refactoring applies to `genBigNumericCmp()` (lines 315-355). |
||||
|
||||
**Expected savings**: ~40 lines. |
||||
|
||||
### 2.2 Data-driven operator dispatch tables |
||||
|
||||
**Files**: `Parser/BinaryOpTrait.php` (16 wrapper methods), `Parser/AssignOpTrait.php` (14 wrapper methods) |
||||
|
||||
30+ thin methods that are just `parseBinaryOp($left, $right, '+')` / `parseAssignOp($node, '+=')`. |
||||
|
||||
**Plan**: Replace with a static map in `parseExpr()`: |
||||
```php |
||||
private const BINARY_OP_MAP = [ |
||||
'Expr_BinaryOp_Plus' => '+', |
||||
'Expr_BinaryOp_Minus' => '-', |
||||
// ... |
||||
]; |
||||
private const ASSIGN_OP_MAP = [ |
||||
'Expr_AssignOp_Plus' => '+=', |
||||
// ... |
||||
]; |
||||
``` |
||||
|
||||
**Expected savings**: ~200 lines removed (boilerplate method bodies). |
||||
|
||||
### 2.3 Deduplicate call dispatch patterns |
||||
|
||||
**Files**: `CompilerBase.php` (`parseFuncCall`, `parseMethodCall`, `parseStaticCall` — ~300 lines combined), `UniversalMethodCall.php` (`tryOptimizePhpFn` vs `dispatchFuncCall`) |
||||
|
||||
These share the same overall flow: resolve callable → try native/optimized path → on `PlaceHolder` fall back to placeholder → parse args → wrap in `php::call()`. Additionally, `tryOptimizePhpFn()` (UniversalMethodCall lines 720-772) duplicates the argument type conversion logic already present in `dispatchFuncCall()` (FuncCallOptimizer lines 234-269). |
||||
|
||||
**Plan**: Extract a shared `resolveCall(CallLike $expr, ...)` method. Unify arg conversion so `tryOptimizePhpFn` delegates to `dispatchFuncCall` instead of reimplementing it. |
||||
|
||||
**Expected savings**: ~40 lines, fixes double-calculation of arg conversions. |
||||
|
||||
### 2.4 Deduplicate UNIVERSAL_METHODS math entries |
||||
|
||||
**File**: `UniversalMethodCall.php` (lines 12-69) |
||||
|
||||
The INT block (lines 12-41) and FLOAT block (lines 42-69) contain 20 identical math method entries (`abs`, `ceil`, `floor`, `sqrt`, `sin`, `cos`, etc.) differing only in `return_type`. Also the `calc_op` entries (add/sub/mul/div) are duplicated. |
||||
|
||||
**Plan**: Define math method names once in a shared array, generate both INT and FLOAT entries in the constructor with the appropriate `return_type`. |
||||
|
||||
**Expected savings**: ~25 lines of config data. |
||||
|
||||
### 2.5 Deduplicate constant folding methods |
||||
|
||||
**File**: `Optimizer/FuncCallOptimizer.php` (lines 515-593) |
||||
|
||||
8 methods (`doFoldStringLen`, `doFoldStringCase`, `doFoldCmp2`, `doFoldCmp3`, `doFoldCountLiteral`, `doFoldKnownClass`, `doFoldKnownConstant`, `doFoldSsaType`) follow the identical pattern: extract args → check types → compute → return literal or false. |
||||
|
||||
**Plan**: Create a generic `tryFold(callable $check, callable $compute)` that handles the arg extraction and short-circuit boilerplate. Each folder becomes a one-liner. |
||||
|
||||
**Expected savings**: ~50 lines. |
||||
|
||||
### 2.6 Remove MSVC compat flag duplication in Clang |
||||
|
||||
**File**: `Backend/Clang.php` |
||||
|
||||
The 4-line MSVC compatibility block (`-fms-compatibility`, `-fms-compatibility-version=19.40`, `-fdelayed-template-parsing`, `-fms-extensions`) appears 7 times (compileFile, buildCompileCommand, buildCCompileCommand, buildNativeCompileCommand, buildFullCompileOptions, buildCompileOptions, buildLinkOptions). |
||||
|
||||
**Plan**: Extract `private function getMsvcCompatFlags(): string` method. Called once per method that needs it instead of repeated inline. |
||||
|
||||
**Expected savings**: ~24 lines, single point of change if MSVC compat flags need updating. |
||||
|
||||
### 2.7 Merge return-check blocks |
||||
|
||||
**File**: `CompilerBase.php`, `parseReturn()` (line 1621) and `genReturnCode()` (line 5741) |
||||
|
||||
Identical 7-line union type check blocks. |
||||
|
||||
**Plan**: Extract `genUnionReturnWrapper(string $exprVar)` method. |
||||
|
||||
**Expected savings**: ~10 lines, eliminates drift risk. |
||||
|
||||
### 2.8 Fix `buildCCompileCommand()` inconsistency between Gcc and Clang |
||||
|
||||
**Files**: `Backend/Gcc.php` (lines 131-137), `Backend/Clang.php` (lines 198-206) |
||||
|
||||
Gcc unconditionally appends `-O$level` then conditionally appends `-g`. Clang treats debug and optimization as mutually exclusive (`if debug: -O0 -g` else `-O$level`). This is a behavioral inconsistency between backends implementing the same abstract method. |
||||
|
||||
**Plan**: Standardize on one behavior (the Clang pattern of `-O0 -g` for debug is the correct one — debug builds should not optimize). This will be automatically resolved by Phase 1.2 (GccLikeBackend). |
||||
|
||||
--- |
||||
|
||||
## Phase 3: Structural Improvements (P2) |
||||
|
||||
### 3.1 Entity flag-check consistency |
||||
|
||||
**File**: `Entity/PropertyDef.php` has `isPrivate()`, `isProtected()`, `isPublic()`, `isStatic()`. `Entity/MethodDef.php` has none — flag checks are done inline in CompilerBase. |
||||
|
||||
**Plan**: Add a `HasFlags` trait used by both `PropertyDef` and `MethodDef`: |
||||
```php |
||||
trait HasFlags { |
||||
public function isPrivate(): bool { return $this->flags & Modifiers::PRIVATE; } |
||||
public function isProtected(): bool { return $this->flags & Modifiers::PROTECTED; } |
||||
public function isPublic(): bool { return !$this->isPrivate() && !$this->isProtected(); } |
||||
public function isStatic(): bool { return $this->flags & Modifiers::STATIC; } |
||||
public function isAbstract(): bool { return $this->flags & Modifiers::ABSTRACT; } |
||||
} |
||||
``` |
||||
|
||||
**Expected savings**: Removes inline flag checks from CompilerBase, adds clarity. |
||||
|
||||
### 3.2 Test infrastructure base class |
||||
|
||||
**Files**: `phpunit/src/AstNodeTypeTest.php`, `CompilerBaseAdapterTest.php`, `TraitsTest.php`, `PreprocessorTest.php` |
||||
|
||||
All 4 duplicate the same setUp/tearDown pattern: create temp dir, `CompilerTest::create()`, recursive cleanup. |
||||
|
||||
**Plan**: Add `CompilerTestCase extends \PHPUnit\Framework\TestCase` to `phpunit/bootstrap.php`: |
||||
```php |
||||
abstract class CompilerTestCase extends TestCase { |
||||
protected string $tmpDir; |
||||
protected CompilerTest $compiler; |
||||
|
||||
protected function setUp(): void { |
||||
parent::setUp(); |
||||
$this->tmpDir = sys_get_temp_dir() . '/compiler_test_' . uniqid(); |
||||
mkdir($this->tmpDir, 0777, true); |
||||
$this->compiler = CompilerTest::create($this->tmpDir); |
||||
} |
||||
|
||||
protected function tearDown(): void { |
||||
parent::tearDown(); |
||||
// recursive cleanup |
||||
} |
||||
} |
||||
``` |
||||
|
||||
### 3.3 Eliminate `buildFull*Options` / `build*Options` duality |
||||
|
||||
**Files**: `Backend/Gcc.php`, `Backend/Clang.php`, `Backend/Msvc.php` |
||||
|
||||
All three backends implement both `buildFullCompileOptions()` / `buildCompileOptions()` and `buildFullLinkOptions()` / `buildLinkOptions()`. The "full" variants are subsets of the "standard" variants working from differently-keyed option arrays. They have drifted independently (e.g., RPATH handling differs between the two in Gcc/Clang). |
||||
|
||||
**Plan**: Make the "full" variants delegate to the "standard" variants by normalizing their option keys once at the call site. Keep only one code path for each (compile/link). |
||||
|
||||
**Expected savings**: ~100+ lines, eliminates drift between the two variants. |
||||
|
||||
### 3.4 Deduplicate Preprocessor AST switch |
||||
|
||||
**File**: `Preprocessor.php` |
||||
|
||||
`prepareFile()` (lines 115-151) and `prepareNamespace()` (lines 196-223) both switch over the same set of AST `Stmt_*` types with nearly identical case bodies. |
||||
|
||||
**Plan**: Extract `processStmt(Node $v)` method that both callers share. |
||||
|
||||
**Expected savings**: ~25 lines. |
||||
|
||||
### 3.5 Remove dead code: `ScopeContext` |
||||
|
||||
**File**: `Context/ScopeContext.php` (7 lines) |
||||
|
||||
An empty class with no properties or methods. Used only as a type annotation in `FunctionContext`. Either populate it with scope-relevant state, or remove it and use plain `\stdClass` / array / null. |
||||
|
||||
### 3.6 Reduce StdContainerTrait coupling |
||||
|
||||
**File**: `Parser/StdContainerTrait.php` (823 lines, 48 methods) |
||||
|
||||
This is effectively a standalone subsystem for std container handling. As a trait, it has unrestricted access to CompilerBase's internals. |
||||
|
||||
**Plan**: Extract core logic into `StdContainerHandler` service class. The trait becomes a thin facade that delegates to the handler. |
||||
|
||||
**Expected savings**: Better testability, clearer boundaries, easier to understand. |
||||
|
||||
--- |
||||
|
||||
## Phase 4: Longer-Term Architectural (P3) |
||||
|
||||
### 4.1 Break CompilerBase into domain-specific classes |
||||
|
||||
Currently CompilerBase is a 5,917-line god class using 20 traits as a workaround for PHP's single inheritance. Consider: |
||||
|
||||
- `ExpressionCompiler` — all parseExpr sub-dispatch (~500 lines) |
||||
- `StatementCompiler` — parseStmts, parseIf, parseWhile, parseFor, parseSwitch, etc. |
||||
- `TypeResolver` — parseTypeDecl, detectClassOfExpr, type checking |
||||
- `CallResolver` — parseFuncCall, parseMethodCall, parseStaticCall, parseNew |
||||
|
||||
These would be injected services rather than traits, making CompilerBase a coordinator. |
||||
|
||||
### 4.2 Shared AST walker pattern with Python Translator |
||||
|
||||
Both PHP and Python translators implement the same "walk-collect-indent-emit" pipeline independently. `Core\Translator` could define a standard `walkAst($nodes, callable $visitor)` that handles indentation and line collection. |
||||
|
||||
--- |
||||
|
||||
## Implementation Order & Impact Matrix |
||||
|
||||
| # | Item | Savings | Risk | Effort | |
||||
|---|------|---------|------|--------| |
||||
| 1.1 | UnixPlatform base class | ~180 dup lines | Low | 2-3h | |
||||
| 1.2 | GccLikeBackend base class | ~250 dup lines | Medium | 3-4h | |
||||
| 2.1 | BigNum dispatch consolidation | ~40 lines | Low | 1h | |
||||
| 2.2 | Data-driven op dispatch | ~200 lines | Low | 1-2h | |
||||
| 2.3 | Unify call dispatch patterns | ~40 lines | Low | 1-2h | |
||||
| 2.4 | UNIVERSAL_METHODS math dedup | ~25 lines | Low | 30m | |
||||
| 2.5 | Fold method template | ~50 lines | Low | 1h | |
||||
| 2.6 | MSVC compat flags in Clang | ~24 lines | Low | 30m | |
||||
| 2.7 | Merge return-check blocks | ~10 lines | Low | 30m | |
||||
| 2.8 | Fix buildCCompileCommand drift | bug fix | Low | 30m | |
||||
| 3.1 | HasFlags trait | clarity | Low | 1h | |
||||
| 3.2 | CompilerTestCase base class | boilerplate | Low | 1h | |
||||
| 3.3 | Eliminate Full*Options duality | ~100 lines | Medium | 2h | |
||||
| 3.4 | Preprocessor AST switch dedup | ~25 lines | Low | 1h | |
||||
| 3.5 | Remove dead ScopeContext | 7 lines | Low | 15m | |
||||
| 3.6 | StdContainer service class | boundary | Medium | 3-4h | |
||||
| 4.1 | Domain classes | architecture | High | 1-2 weeks | |
||||
| 4.2 | AST walker pattern | architecture | Medium | 3-5h | |
||||
|
||||
**Total estimated savings**: ~950+ lines of duplicated / dead code. |
||||
|
||||
**Recommended execution**: Phase 1 → Phase 2 → Phase 3. Items within each phase are independent and can be parallelized. |
||||
@ -1,325 +0,0 @@ |
||||
# Encapsulation Review |
||||
|
||||
## Summary |
||||
|
||||
| Metric | Value | |
||||
|--------|-------| |
||||
| Entity classes with all-public fields | 9/9 (100%) | |
||||
| FunctionContext public properties | 26 (all mutable) | |
||||
| CompilerBase private methods | 5/269 (1.9%) | |
||||
| Traits with direct `$this->context->` access | 6 traits, 60+ sites | |
||||
| ScopeContext (dead code) | 7 lines, empty class | |
||||
|
||||
--- |
||||
|
||||
## Issue 1: Entity classes — all-public mutable fields |
||||
|
||||
**Severity**: High. Every entity class exposes all internal state as public writable properties. External code in Preprocessor/CompilerBase directly mutates them. |
||||
|
||||
### 1.1 ClassDef (18 public properties) |
||||
|
||||
`src/Php/Entity/ClassDef.php` |
||||
|
||||
```php |
||||
public array $methods = []; // externally populated: $classDef->properties[$name] = ... |
||||
public array $properties = []; // externally populated |
||||
public array $constants = []; // externally populated |
||||
public array $implements = []; // externally populated |
||||
public string $extends = ''; // externally set: $this->classDef->extends = ... |
||||
public bool $requireCtor = false; |
||||
public bool $enum = false; |
||||
public ?string $enumBackingType = null; |
||||
public array $enumCases = []; |
||||
public array $abstractMethods = []; |
||||
public ?Trait_ $trait = null; |
||||
public array $traitAliases = []; |
||||
public array $traitIgnored = []; |
||||
public int $flags; // no visibility checks, raw bitmask |
||||
public bool $inheritedFromInternalClass = false; |
||||
public string $ctorInit = ''; // mutated during code generation |
||||
public string $ctorClean = ''; // mutated during code generation |
||||
public FunctionContext $propertyContext; // set after construction |
||||
``` |
||||
|
||||
**Issues**: |
||||
- `$properties`, `$methods`, `$constants` — exposed as raw arrays. External code does `$classDef->properties[$name] = $propDef`. No validation that the key matches `$propDef->name`, no type enforcement. |
||||
- `$flags` — raw int, no guarantee it's a valid Modifiers bitmask. |
||||
- `$ctorInit` / `$ctorClean` — mutated by CompilerBase during code generation, not initialization. |
||||
- Property additions use `addMethod()`, `addAbstractMethod()` but array properties are also set directly via `[] =`. |
||||
- `$extends` — set directly as raw string, bypasses `parent::__construct()` which also sets it on ClassLikeDef. |
||||
|
||||
**Recommendation**: |
||||
- Make `$methods`, `$properties`, `$constants` private, expose via `addMethod()`/`getMethod()` (already exists) |
||||
- Make `$flags` private, expose `isAbstract()` (already exists), add `isFinal()`, `isReadonly()` |
||||
- Make `$extends` write-once via `setExtends(string)` with validation |
||||
- Add `appendCtorInit(string)` and `appendCtorClean(string)` methods instead of direct string mutation |
||||
|
||||
### 1.2 FunctionDef (12 public properties) |
||||
|
||||
`src/Php/Entity/FunctionDef.php` |
||||
|
||||
```php |
||||
public string $name; |
||||
public string $returnType; |
||||
public array $argInfoList = []; // externally populated: $functionDef->argInfoList[] = $argInfo |
||||
public int $argCountRequired = 0; |
||||
public string $params = ''; // generated C++ param string, mutated during compilation |
||||
public string $namespace; |
||||
public bool $method = false; |
||||
public bool $stub = false; |
||||
public bool $returnTypeUndeclared = false; |
||||
public string $returnClass = ''; |
||||
public ?array $returnTypeCheck = null; |
||||
public string $returnTypeStr = ''; |
||||
public ?NodeAbstract $returnTypeNode = null; |
||||
``` |
||||
|
||||
**Issues**: |
||||
- `$name` and `$namespace` are set in constructor but still publicly writable — should be readonly |
||||
- `$argInfoList[]` is directly appended to by Preprocessor (line 329) |
||||
- `$params` is a codegen artifact stored on the entity — belongs in a separate compilation context |
||||
|
||||
**Recommendation**: |
||||
- Make constructor-set properties readonly (`$name`, `$namespace`, `$returnType`) |
||||
- Add `addArg(ArgInfo $arg)` method instead of direct array mutation |
||||
- Extract `$params` to a compilation context separate from the definition entity |
||||
|
||||
### 1.3 PropertyDef (7 public properties) |
||||
|
||||
`src/Php/Entity/PropertyDef.php` |
||||
|
||||
```php |
||||
public string $name; |
||||
public string $type; |
||||
public int $flags; |
||||
public ?string $default = null; |
||||
public ?ArrayInitPlan $arrayInitPlan = null; |
||||
public bool $nullable = false; |
||||
public string $class = ''; // set after construction |
||||
``` |
||||
|
||||
**Issues**: |
||||
- `$class` is set after construction externally (`$propDef->class = $fullClassName`) |
||||
- `$flags` is raw int — already has `isPrivate()`/`isProtected()`/`isPublic()`/`isStatic()` methods, good |
||||
- Constructor already sets all core fields — `$class` should be added to the constructor |
||||
|
||||
**Recommendation**: |
||||
- Add `$class` to the constructor (it's always known at construction time) |
||||
- Make constructor-set fields readonly or private |
||||
|
||||
### 1.4 MethodDef (4 public properties) |
||||
|
||||
`src/Php/Entity/MethodDef.php` |
||||
|
||||
```php |
||||
public int $flags; |
||||
public string $name; |
||||
public ?FunctionDef $functionDef = null; // set after construction |
||||
public bool $hasDynamicCall = false; |
||||
``` |
||||
|
||||
**Issues**: |
||||
- No flag-check methods — inline checks in CompilerBase should use `$methodDef->isPrivate()` instead |
||||
- `$functionDef` is set externally: `$this->methodDef->functionDef = $functionDef` (Preprocessor line 411) |
||||
|
||||
**Recommendation**: |
||||
- Add `HasFlags` trait (from Phase 3.1 of reuse plan) |
||||
- Add `setFunctionDef(FunctionDef $fd)` method with validation |
||||
|
||||
### 1.5 ConstantDef (8 public properties) |
||||
|
||||
```php |
||||
public string $name; |
||||
public string $type; |
||||
public int $flags; |
||||
public string $value; |
||||
public string $arrayExpr = ''; |
||||
public string $class = ''; |
||||
public ?NodeAbstract $valueExpr = null; |
||||
``` |
||||
|
||||
**Issues**: Same pattern — constructor sets core fields, but `$class` is set externally afterward. |
||||
|
||||
**Recommendation**: Add `$class` to constructor. |
||||
|
||||
--- |
||||
|
||||
## Issue 2: FunctionContext — public mutable grab-bag |
||||
|
||||
**Severity**: High. 26 public properties, all writable by any code with access to the context object. |
||||
|
||||
`src/Php/Context/FunctionContext.php` |
||||
|
||||
```php |
||||
public ?SsaBuilder $ssaBuilder = null; // transient analysis state |
||||
public array $stableObjects = []; // SSA optimizer state |
||||
public array $hoistedProps = []; // SSA optimizer state |
||||
public array $unsafeObjectProps = []; // SSA optimizer state |
||||
public array $objects = []; // object variable tracking |
||||
public array $stdArrays = []; // std container tracking |
||||
public array $stdContainers = []; // std container tracking |
||||
public array $localVars = []; // local variable table |
||||
public array $staticVars = []; // static variable table |
||||
public array $globalVars = []; // global variable table |
||||
public array $ceWrappers = []; // class entry wrappers |
||||
public int $tmpVarIndex = 0; // auto-increment counter |
||||
public array $arguments = []; // function arguments |
||||
public bool $inLoop = false; // control-flow state |
||||
public bool $inClosure = false; // control-flow state |
||||
public bool $hasMultiLevelBreak = false; |
||||
public bool $hasMultiLevelContinue = false; |
||||
public bool $inAssignExpr = false; // expression context |
||||
public array $beforeStmtLines = []; // deferred code (flushed before stmts) |
||||
public array $afterStmtLines = []; // deferred code (flushed after stmts) |
||||
public array $objectProps; // (uninitialized!) |
||||
public array $staticPropRefs = []; // static property references |
||||
public int $scopeLevel = 0; // lexical scope depth |
||||
/** @var array<int, ScopeContext> */ |
||||
public array $scopeLayouts = []; // per-scope data |
||||
``` |
||||
|
||||
**Issues**: |
||||
- Traits directly mutate deeply nested state: `$this->context->stdArrays[$var] = ...`, `$this->context->localVars[$name] = ...` |
||||
- No semantic grouping — analysis state, variable tracking, control-flow flags all mixed |
||||
- `$objectProps` is declared but never initialized (could be null at runtime) |
||||
- `$scopeLayouts` is managed through `enterScope()`/`leaveScope()` — but can be bypassed |
||||
- `$tmpVarIndex` auto-increment — should use a method instead of direct `++` |
||||
|
||||
**Recommendation**: |
||||
- Group related properties into sub-objects: `VariableTable`, `ControlFlowState`, `ScopeManager` |
||||
- Make properties that should only be read by the compiler layer private/protected with getters |
||||
- Add `incrementTmpVar(): int`, `addLocalVar()`, `addBeforeStmt()` methods |
||||
- Initialize `$objectProps = []` |
||||
|
||||
--- |
||||
|
||||
## Issue 3: CompilerBase — only 1.9% private methods |
||||
|
||||
**Severity**: Medium. Virtually everything is public or protected. |
||||
|
||||
`src/Php/CompilerBase.php` — 269 methods total: |
||||
- ~25 public methods (many should be protected or internal) |
||||
- ~239 protected methods (most should be private — internal helpers) |
||||
- **5 private methods** (1.9%) |
||||
|
||||
### 3.1 Methods that should be private |
||||
|
||||
The following methods are internal helpers only called from within CompilerBase (not from Preprocessor, Translator, or traits). They are unnecessarily `protected`: |
||||
|
||||
| Method | Line | Called from | |
||||
|--------|------|-------------| |
||||
| `resetFunction()` | 833 | Internal only | |
||||
| `resetMethod()` | 840 | Internal only | |
||||
| `resetClass()` | 846 | Internal only | |
||||
| `resolveObjectClassDef()` | 816 | Already private ✓ | |
||||
| `getBigIntLiteralString()` | 1183 | Already private ✓ | |
||||
| `getDecimalLiteralString()` | 1188 | Already private ✓ | |
||||
| `parseBeforeStmtLines()` | 1321 | Internal, but accessed by traits | |
||||
| `parseAfterStmtLines()` | 1331 | Internal, but accessed by traits | |
||||
| `genTmpVarName()` | 734 | Public — should at least be protected | |
||||
|
||||
### 3.2 Public methods that are internal concern |
||||
|
||||
| Method | Current visibility | Issue | |
||||
|--------|-------------------|-------| |
||||
| `genTmpVarName()` | public | Only used internally for variable name generation | |
||||
| `writeFile()` | public | File I/O — should be a separate service | |
||||
| `stop()` | public | Error helper — could be internal | |
||||
| `isScalarInt()` | public | AST helper, only used internally | |
||||
| `getType()` | public | AST helper, only used internally | |
||||
| `getObjectType()` | public | Type mapping, used internally | |
||||
| `getTypeFromZendType()` | public | Type mapping, used internally | |
||||
| `getIncludeDir()` | public | Config getter — should be on a Config object | |
||||
| `getBuildDir()` | public | Config getter — should be on a Config object | |
||||
|
||||
### 3.3 Public constants leaked as API |
||||
|
||||
30 public constants for internal type names, literal values, etc. These are needed by traits but expose internal naming conventions. |
||||
|
||||
--- |
||||
|
||||
## Issue 4: Trait → Context coupling |
||||
|
||||
**Severity**: Medium. Traits bypass any encapsulation boundary and directly mutate `$this->context`. |
||||
|
||||
| Trait | `$this->context->` accesses | |
||||
|-------|---------------------------| |
||||
| `StdContainerTrait` | 40+ accesses to `stdArrays`, `stdContainers`, `objects`, `localVars` | |
||||
| `LoopVarOptimizer` | accesses to `localVars`, `arguments`, `scopeLevel`, `inLoop` | |
||||
| `SsaPropOptimizer` | accesses to `stableObjects`, `hoistedProps`, `unsafeObjectProps`, `objects` | |
||||
| `FuncCallOptimizer` | accesses to `beforeStmtLines`, `arguments`, `localVars` | |
||||
| `SsaTypeOptimizer` | accesses to `localVars` | |
||||
| `BinaryOpTrait` | accesses to `objects`, `localVars` | |
||||
|
||||
**Issues**: |
||||
- Traits have no declared contract — they assume `$this->context` exists and has specific properties |
||||
- If a property name changes in FunctionContext, all 6 traits break silently |
||||
- No type safety — arrays are indexed by string but accessed with arbitrary keys |
||||
|
||||
**Recommendation**: |
||||
- Define a `ContextAccess` interface that traits must use instead of direct property access |
||||
- Or: inject context into trait methods as a parameter instead of reading from `$this` |
||||
- Short-term: add `@property-read` annotations to document the contract |
||||
|
||||
--- |
||||
|
||||
## Issue 5: Preprocessor directly mutates entity state |
||||
|
||||
**Severity**: Medium. Preprocessor bypasses entity boundaries. |
||||
|
||||
`src/Php/Preprocessor.php`: |
||||
```php |
||||
line 279: $argInfo->name = $name; // direct property set |
||||
line 329: $functionDef->argInfoList[] = $argInfo; // direct array append |
||||
line 411: $this->methodDef->functionDef = $functionDef; // direct property set |
||||
line 447: $this->classDef->extends = $this->parentClass; // direct property set |
||||
line 585: $this->classDef->constants[$constInfo->name] = ...; // direct array set |
||||
line 618: $this->classDef->properties[$name] = $propDef; // direct array set |
||||
``` |
||||
|
||||
**Recommendation**: Use entity methods: `$functionDef->addArg($argInfo)`, `$this->methodDef->setFunctionDef($functionDef)`, `$classDef->addProperty($propDef)`, etc. |
||||
|
||||
--- |
||||
|
||||
## Issue 6: CompilerBase protected state leaked to inheritance chain |
||||
|
||||
**Severity**: Low-Medium. The chain CompilerBase → Preprocessor → Translator means any protected property in CompilerBase is accessible from Translator. |
||||
|
||||
CompilerBase has ~50 protected properties. Translator is 3301 lines and accesses many of them. There's no way to know which properties are "safe to use" vs "internal to CompilerBase." |
||||
|
||||
**Recommendation**: Migrate internal-only properties to `private` over time, with explicit getter methods where needed. |
||||
|
||||
--- |
||||
|
||||
## Issue 7: Platform/Backend — well encapsulated |
||||
|
||||
**Severity**: None. The Platform and Backend layers are well-encapsulated: |
||||
- All state is private (e.g., `$compilerCommand`, `$linkerCommand` in GccLikeBackend) |
||||
- Only methods are public |
||||
- Abstract contracts are clear |
||||
- Factory pattern is used consistently |
||||
|
||||
**No changes needed in this layer.** |
||||
|
||||
--- |
||||
|
||||
## Issue 8: ScopeContext is dead code |
||||
|
||||
**Severity**: Low. `src/Php/Context/ScopeContext.php` — 7 lines, empty class body. Used as a placeholder type in FunctionContext's `$scopeLayouts` array. Either populate it or remove it. |
||||
|
||||
--- |
||||
|
||||
## Implementation Priority |
||||
|
||||
| # | Issue | Impact | Effort | Risk | |
||||
|---|-------|--------|--------|------| |
||||
| 1.1 | Entity: readonly for constructor fields | Data integrity | 2h | Low | |
||||
| 1.2 | Entity: add mutation methods (addArg, addProperty, etc.) | Safe mutation | 3h | Medium | |
||||
| 2.1 | FunctionContext: group properties into sub-objects | Clarity | 4h | Medium | |
||||
| 2.2 | FunctionContext: add accessor methods | Controlled mutation | 3h | Medium | |
||||
| 3 | CompilerBase: demote public→protected, protected→private | Boundary clarity | 4h | Medium | |
||||
| 4 | Define trait context contract | Safe coupling | 3h | Medium | |
||||
| 5 | Preprocessor: use entity methods | Consistent mutation | 2h | Low | |
||||
| 8 | Remove ScopeContext dead code | Cleanup | 15m | None | |
||||
|
||||
**Recommended order**: Start with 8 (quick win), then 1.1 + 1.2 (entity cleanup), then 2.1 + 2.2 (context cleanup), then 3 + 5 + 4 (CompilerBase boundary). |
||||
@ -0,0 +1,21 @@ |
||||
name: Patch PHP headers for C++ |
||||
description: Apply temporary upstream PHP header fixes required by generated C++. |
||||
|
||||
runs: |
||||
using: composite |
||||
steps: |
||||
# Temporary workaround for php/php-src#22935. Some supported PHP packages |
||||
# contain a php_hash.h revision that is valid C but invalid C++. Remove |
||||
# this action when all PHP 8.4/8.5 packages include php/php-src#22940. |
||||
- name: Patch php_hash.h |
||||
shell: bash |
||||
run: | |
||||
php_include_dir="$(php-config --include-dir)" |
||||
php_hash_header="${php_include_dir}/ext/hash/php_hash.h" |
||||
|
||||
if grep -Fq 'char *base = ecalloc(' "${php_hash_header}"; then |
||||
sudo patch --directory="${php_include_dir}" --strip=1 \ |
||||
< "${GITHUB_ACTION_PATH}/../../patches/php-hash-cxx.patch" |
||||
else |
||||
echo "php_hash.h already contains the upstream C++ fix" |
||||
fi |
||||
@ -0,0 +1,155 @@ |
||||
name: Unix ARM64 build |
||||
description: Build and smoke-test TypePHP on a Unix ARM64 runner |
||||
|
||||
inputs: |
||||
php-version: |
||||
description: PHP minor version |
||||
required: true |
||||
os: |
||||
description: Package operating-system identifier |
||||
required: true |
||||
library-extension: |
||||
description: Shared-library extension used by PHPX |
||||
required: true |
||||
smoke-directory: |
||||
description: Platform smoke project directory below .github/smoke |
||||
required: true |
||||
smoke-binary: |
||||
description: Platform smoke executable name |
||||
required: true |
||||
smoke-output: |
||||
description: Expected platform smoke output |
||||
required: true |
||||
|
||||
runs: |
||||
using: composite |
||||
steps: |
||||
- name: Setup PHP |
||||
uses: shivammathur/setup-php@v2 |
||||
with: |
||||
php-version: ${{ inputs.php-version }} |
||||
coverage: none |
||||
extensions: mbstring |
||||
ini-values: precision=17, memory_limit=4G, error_reporting=E_ERROR|E_WARNING, display_errors=1, display_startup_errors=1, log_errors=0 |
||||
tools: composer:v2 |
||||
env: |
||||
fail-fast: true |
||||
phpts: ts |
||||
update: true |
||||
|
||||
- name: Install Linux build dependencies |
||||
if: inputs.os == 'linux' |
||||
shell: bash |
||||
run: | |
||||
sudo apt-get update |
||||
sudo apt-get install --yes build-essential cmake libgmp-dev libmpfr-dev pkg-config |
||||
|
||||
- name: Install macOS build dependencies |
||||
if: inputs.os == 'macos' |
||||
shell: bash |
||||
run: brew install cmake gmp mpfr pkg-config |
||||
|
||||
- name: Verify ZTS PHP and architecture |
||||
shell: bash |
||||
run: | |
||||
php -r 'if (!PHP_ZTS) { fwrite(STDERR, "Expected a ZTS PHP build\n"); exit(1); }' |
||||
case "$(php -r 'echo PHP_VERSION;')" in |
||||
"${{ inputs.php-version }}"*) ;; |
||||
*) echo "setup-php installed an unexpected PHP version" >&2; exit 1 ;; |
||||
esac |
||||
case "$(uname -m)" in |
||||
arm64|aarch64) ;; |
||||
*) echo "Expected an ARM64 runner, got $(uname -m)" >&2; exit 1 ;; |
||||
esac |
||||
|
||||
php_home="$(php-config --prefix)" |
||||
embed_library="${php_home}/lib/libphp.${{ inputs.library-extension }}" |
||||
test -x "${php_home}/bin/php-config" |
||||
test -f "${embed_library}" |
||||
echo "PHP_HOME=${php_home}" >> "${GITHUB_ENV}" |
||||
if [[ '${{ inputs.os }}' == 'linux' ]]; then |
||||
echo "LD_LIBRARY_PATH=${PHPX_HOME}/lib:${php_home}/lib" >> "${GITHUB_ENV}" |
||||
else |
||||
echo "DYLD_LIBRARY_PATH=${PHPX_HOME}/lib:${php_home}/lib" >> "${GITHUB_ENV}" |
||||
fi |
||||
php -v |
||||
php-config --configure-options |
||||
|
||||
- name: Patch php_hash.h C++ compatibility |
||||
uses: ./.github/actions/patch-php-headers |
||||
|
||||
- name: Install Composer dependencies |
||||
shell: bash |
||||
run: composer install --prefer-dist --no-progress |
||||
|
||||
- name: Build PHPX |
||||
shell: bash |
||||
run: | |
||||
cmake -S "${PHPX_HOME}" -B "${PHPX_HOME}/build" \ |
||||
-D CMAKE_BUILD_TYPE=Release \ |
||||
-D BUILD_TESTS=OFF \ |
||||
-D BUILD_EXT=OFF \ |
||||
-D GITHUB_ACTION=ON \ |
||||
-D php_dir="${PHP_HOME}" |
||||
cmake --build "${PHPX_HOME}/build" --target phpx --parallel 2 |
||||
test -f "${PHPX_HOME}/lib/libphpx.${{ inputs.library-extension }}" |
||||
|
||||
- name: Build tpc |
||||
shell: bash |
||||
run: | |
||||
php bin/tpc.php project.yml --job 2 --no-progress |
||||
test -x ./tpc |
||||
file ./tpc |
||||
file ./tpc | grep -Eiq 'arm64|aarch64|ARM aarch64' |
||||
./tpc --version |
||||
|
||||
- name: Run platform smoke test |
||||
shell: bash |
||||
run: | |
||||
smoke_root=".github/smoke/${{ inputs.smoke-directory }}" |
||||
./tpc "${smoke_root}/project.yml" --job 1 --no-progress |
||||
smoke_binary="${smoke_root}/${{ inputs.smoke-binary }}" |
||||
test -x "${smoke_binary}" |
||||
output="$("${smoke_binary}")" |
||||
test "${output}" = '${{ inputs.smoke-output }}' |
||||
|
||||
- name: Show native dependencies |
||||
shell: bash |
||||
run: | |
||||
if [[ '${{ inputs.os }}' == 'linux' ]]; then |
||||
ldd ./tpc |
||||
else |
||||
otool -L ./tpc |
||||
fi |
||||
|
||||
- name: Package tested compiler |
||||
if: startsWith(github.ref, 'refs/tags/') && inputs.php-version == '8.5' |
||||
shell: bash |
||||
run: | |
||||
composer install --no-dev --prefer-dist --no-progress --classmap-authoritative |
||||
export TYPEPHP_PACKAGE_VERSION="${GITHUB_REF_NAME}" |
||||
php package.php |
||||
test "$(find . -maxdepth 1 -name 'tpc_v*_${{ inputs.os }}_arm64.tar.gz' -type f | wc -l)" -eq 1 |
||||
|
||||
- name: Upload release package |
||||
if: startsWith(github.ref, 'refs/tags/') && inputs.php-version == '8.5' |
||||
uses: actions/upload-artifact@v4 |
||||
with: |
||||
name: release-${{ inputs.os }}-arm64-php-${{ inputs.php-version }}-zts |
||||
if-no-files-found: error |
||||
retention-days: 1 |
||||
path: tpc_v*_${{ inputs.os }}_arm64.tar.gz |
||||
|
||||
- name: Upload platform build outputs |
||||
if: always() |
||||
uses: actions/upload-artifact@v4 |
||||
with: |
||||
name: tpc-${{ inputs.os }}-arm64-php-${{ inputs.php-version }}-zts |
||||
include-hidden-files: true |
||||
if-no-files-found: warn |
||||
retention-days: 7 |
||||
path: | |
||||
tpc |
||||
.github/smoke/${{ inputs.smoke-directory }}/${{ inputs.smoke-binary }} |
||||
.github/smoke/${{ inputs.smoke-directory }}/build/**/*.cc |
||||
.github/smoke/${{ inputs.smoke-directory }}/build/**/*.h |
||||
@ -1,90 +0,0 @@ |
||||
# Copilot instructions for this repository |
||||
|
||||
## Project overview |
||||
|
||||
TypePHP is a PHP native compilation project. Its `tpc` command is TypePHP Compiler (AOT), which translates PHP source into C++, then compiles and links it into a native binary or a PHP extension. The primary entrypoint boots `src/compiler.php`; that drives `TypePhp\Translator` through a fixed pipeline: |
||||
|
||||
1. `prepare()` scans files, parses ASTs, collects symbols, and topologically sorts PHP files by cross-file symbol usage. |
||||
2. `convert()` turns PHP ASTs into generated `.cc` files while passing through native source files (`.cpp`, `.c`, `.s`, `.m`, `.mm`). |
||||
3. `compile()` chooses the platform/compiler backend, generates support sources and headers, and compiles sources, using `pcntl` parallelism when available. |
||||
4. `build()` links object files into the final executable or extension. |
||||
|
||||
`src/CompilerBase.php` contains most PHP-to-C++ translation logic and mixes in many traits for syntax handling and optimizations. `src/Preprocessor.php` owns dependency discovery and file ordering. Platform-specific behavior lives under `src/Platform/`, compiler backends under `src/Backend/`, and metadata/state objects under `src/Entity/` and `src/Context/`. |
||||
|
||||
## Setup and build commands |
||||
|
||||
The repo expects PHP 8.4+, GCC 9+ with C++17, CMake 3.24+, and a compiled `swoole/phpx` dependency. Install PHP dependencies with: |
||||
|
||||
```bash |
||||
composer install |
||||
``` |
||||
|
||||
Build `phpx` before relying on compiler runs: |
||||
|
||||
```bash |
||||
cd vendor/swoole/phpx |
||||
cmake . |
||||
make -j32 |
||||
``` |
||||
|
||||
Compile a project, directory, single file, or `project.yml`: |
||||
|
||||
```bash |
||||
./tpc <path-to-project-or-file> |
||||
./tpc <path> -O2 |
||||
./tpc <path> --mode=ext -o <output_name> |
||||
``` |
||||
|
||||
## Test commands |
||||
|
||||
Run the PHPUnit suite: |
||||
|
||||
```bash |
||||
./vendor/bin/phpunit |
||||
``` |
||||
|
||||
Run a single PHPUnit file or a single test method: |
||||
|
||||
```bash |
||||
./vendor/bin/phpunit phpunit/src/Platform/PlatformTest.php |
||||
./vendor/bin/phpunit --filter testWindowsBasic phpunit/src/Platform/PlatformTest.php |
||||
``` |
||||
|
||||
Run PHPT integration tests: |
||||
|
||||
```bash |
||||
php run-tests.php tests/compiler/ |
||||
php run-tests.php tests/compiler/arrays.phpt |
||||
``` |
||||
|
||||
For parser/runtime comparison without AOT compilation, there are docs using: |
||||
|
||||
```bash |
||||
php run-tests.php --no-aot tests/compiler/arrow-functions.phpt |
||||
``` |
||||
|
||||
## Formatting |
||||
|
||||
The repo ships a PHP CS Fixer config in `.php-cs-fixer.dist.php`: |
||||
|
||||
```bash |
||||
php vendor/bin/php-cs-fixer fix --config=.php-cs-fixer.dist.php <path> |
||||
``` |
||||
|
||||
Generated C++ is auto-formatted by the compiler itself when `clang-format` is available. |
||||
|
||||
## Configuration and repository conventions |
||||
|
||||
- `project.yml` is the project-level build config. Important keys include `name`, `build-mode`, `cxx-std`, `cxx-flags`, `ld-flags`, `sources`, `ignore`, and `resource`. |
||||
- Command-line options intentionally override YAML values. `Translator` parses YAML first, then applies CLI arguments last. |
||||
- YAML parsing accepts both hyphenated and underscored variants for several keys, but existing examples use hyphenated names such as `build-mode` and `cxx-std`. |
||||
- In `bin` mode, compiled programs must define `main()`. In `ext` mode they do not. |
||||
- File discovery is mixed-language by design: PHP is translated, while native sources are compiled directly if they appear in configured sources. |
||||
- Generated files are written under `build/`, with generated C++ paths mirroring the source tree and generated headers under `build/include/`. |
||||
- Platform/compiler selection is centralized: `PlatformFactory` detects the OS, and `CompilerFactory` picks the backend (`Gcc`, `Clang`, `Msvc`) with environment/config overrides. |
||||
|
||||
## Test-specific conventions |
||||
|
||||
- PHPUnit tests for compiler internals should use `CompilerTest::create(ROOT_PATH)`, which enables test mode instead of normal fatal exits. |
||||
- `phpunit/bootstrap.php` exposes a `BaseTest::exec()` helper that expects compilation failures to surface as `TypePhp\Exception\TestError`. |
||||
- PHPT end-to-end tests live in `tests/compiler/`; existing guidance and examples generally put executable test logic inside a `main()` function. |
||||
@ -0,0 +1,11 @@ |
||||
--- a/ext/hash/php_hash.h
|
||||
+++ b/ext/hash/php_hash.h
|
||||
@@ -158,7 +158,7 @@ static inline void *php_hash_alloc_context(const php_hash_ops *ops) {
|
||||
/* Zero out context memory so serialization doesn't expose internals */
|
||||
if (ops->context_align > 0) {
|
||||
size_t align = ops->context_align;
|
||||
- char *base = ecalloc(1, ops->context_size + align);
|
||||
+ char *base = (char *) ecalloc(1, ops->context_size + align);
|
||||
size_t offset = align - ((uintptr_t)base & (align - 1));
|
||||
char *ptr = base + offset;
|
||||
ptr[-1] = (char)offset;
|
||||
@ -0,0 +1,44 @@ |
||||
#include <phpx.h> |
||||
|
||||
#include <cstring> |
||||
#include <sys/utsname.h> |
||||
#include <unistd.h> |
||||
|
||||
using namespace php; |
||||
|
||||
Int php_linux_current_process_id() |
||||
{ |
||||
return static_cast<Int>(getpid()); |
||||
} |
||||
|
||||
Int php_linux_online_processor_count() |
||||
{ |
||||
return static_cast<Int>(sysconf(_SC_NPROCESSORS_ONLN)); |
||||
} |
||||
|
||||
Bool php_linux_uname_machine_is_arm64() |
||||
{ |
||||
utsname info{}; |
||||
if (uname(&info) != 0) { |
||||
return false; |
||||
} |
||||
return std::strcmp(info.machine, "aarch64") == 0 || std::strcmp(info.machine, "arm64") == 0; |
||||
} |
||||
|
||||
Bool php_linux_native_is_arm64() |
||||
{ |
||||
#if defined(__aarch64__) || defined(__arm64__) |
||||
return true; |
||||
#else |
||||
return false; |
||||
#endif |
||||
} |
||||
|
||||
Bool php_linux_native_php_is_zts() |
||||
{ |
||||
#ifdef ZTS |
||||
return true; |
||||
#else |
||||
return false; |
||||
#endif |
||||
} |
||||
@ -0,0 +1,12 @@ |
||||
<?php |
||||
|
||||
/** Linux ARM64 platform declarations implemented by platform.cc. */ |
||||
function linux_current_process_id(): int {} |
||||
|
||||
function linux_online_processor_count(): int {} |
||||
|
||||
function linux_uname_machine_is_arm64(): bool {} |
||||
|
||||
function linux_native_is_arm64(): bool {} |
||||
|
||||
function linux_native_php_is_zts(): bool {} |
||||
@ -0,0 +1,22 @@ |
||||
<?php |
||||
|
||||
function requireLinuxArm64Condition(bool $condition, string $message): void |
||||
{ |
||||
if (!$condition) { |
||||
throw new RuntimeException($message); |
||||
} |
||||
} |
||||
|
||||
function main(): void |
||||
{ |
||||
requireLinuxArm64Condition(PHP_OS_FAMILY === 'Linux', 'Expected PHP_OS_FAMILY=Linux'); |
||||
requireLinuxArm64Condition(DIRECTORY_SEPARATOR === '/', 'Expected the Unix directory separator'); |
||||
requireLinuxArm64Condition(PHP_ZTS !== 0 && PHP_ZTS !== false, 'Expected a ZTS PHP runtime'); |
||||
requireLinuxArm64Condition(linux_native_php_is_zts(), 'The native ZTS macro is not enabled'); |
||||
requireLinuxArm64Condition(linux_native_is_arm64(), 'The native compiler target is not ARM64'); |
||||
requireLinuxArm64Condition(linux_current_process_id() > 0, 'getpid() failed'); |
||||
requireLinuxArm64Condition(linux_online_processor_count() > 0, 'sysconf() returned no processors'); |
||||
requireLinuxArm64Condition(linux_uname_machine_is_arm64(), 'uname() did not report ARM64'); |
||||
|
||||
echo 'linux-arm64-smoke-ok:zts'; |
||||
} |
||||
@ -0,0 +1,9 @@ |
||||
name: linux-arm64-smoke |
||||
mode: bin |
||||
build-dir: build |
||||
output: linux_arm64_smoke |
||||
cxx-std: c++17 |
||||
|
||||
sources: |
||||
- main.php |
||||
- cpp-src |
||||
@ -0,0 +1,45 @@ |
||||
#include <phpx.h> |
||||
|
||||
#include <mach/mach.h> |
||||
#include <sys/sysctl.h> |
||||
#include <unistd.h> |
||||
|
||||
using namespace php; |
||||
|
||||
Int php_macos_current_process_id() |
||||
{ |
||||
return static_cast<Int>(getpid()); |
||||
} |
||||
|
||||
Int php_macos_logical_processor_count() |
||||
{ |
||||
int count = 0; |
||||
size_t size = sizeof(count); |
||||
if (sysctlbyname("hw.logicalcpu", &count, &size, nullptr, 0) != 0) { |
||||
return 0; |
||||
} |
||||
return static_cast<Int>(count); |
||||
} |
||||
|
||||
Bool php_macos_has_mach_host_port() |
||||
{ |
||||
return mach_host_self() != MACH_PORT_NULL; |
||||
} |
||||
|
||||
Bool php_macos_native_is_arm64() |
||||
{ |
||||
#if defined(__aarch64__) || defined(__arm64__) |
||||
return true; |
||||
#else |
||||
return false; |
||||
#endif |
||||
} |
||||
|
||||
Bool php_macos_native_php_is_zts() |
||||
{ |
||||
#ifdef ZTS |
||||
return true; |
||||
#else |
||||
return false; |
||||
#endif |
||||
} |
||||
@ -0,0 +1,12 @@ |
||||
<?php |
||||
|
||||
/** macOS platform declarations implemented by platform.cc. */ |
||||
function macos_current_process_id(): int {} |
||||
|
||||
function macos_logical_processor_count(): int {} |
||||
|
||||
function macos_has_mach_host_port(): bool {} |
||||
|
||||
function macos_native_is_arm64(): bool {} |
||||
|
||||
function macos_native_php_is_zts(): bool {} |
||||
@ -0,0 +1,22 @@ |
||||
<?php |
||||
|
||||
function requireMacosCondition(bool $condition, string $message): void |
||||
{ |
||||
if (!$condition) { |
||||
throw new RuntimeException($message); |
||||
} |
||||
} |
||||
|
||||
function main(): void |
||||
{ |
||||
requireMacosCondition(PHP_OS_FAMILY === 'Darwin', 'Expected PHP_OS_FAMILY=Darwin'); |
||||
requireMacosCondition(DIRECTORY_SEPARATOR === '/', 'Expected the Unix directory separator'); |
||||
requireMacosCondition(PHP_ZTS !== 0 && PHP_ZTS !== false, 'Expected a ZTS PHP runtime'); |
||||
requireMacosCondition(macos_native_php_is_zts(), 'The native ZTS macro is not enabled'); |
||||
requireMacosCondition(macos_native_is_arm64(), 'The native compiler target is not ARM64'); |
||||
requireMacosCondition(macos_current_process_id() > 0, 'getpid() failed'); |
||||
requireMacosCondition(macos_logical_processor_count() > 0, 'sysctl() returned no processors'); |
||||
requireMacosCondition(macos_has_mach_host_port(), 'mach_host_self() failed'); |
||||
|
||||
echo 'macos-arm64-smoke-ok:zts'; |
||||
} |
||||
@ -0,0 +1,9 @@ |
||||
name: macos-smoke |
||||
mode: bin |
||||
build-dir: build |
||||
output: macos_smoke |
||||
cxx-std: c++17 |
||||
|
||||
sources: |
||||
- main.php |
||||
- cpp-src |
||||
@ -0,0 +1,43 @@ |
||||
name: Linux ARM64 |
||||
|
||||
on: |
||||
push: |
||||
pull_request: |
||||
workflow_dispatch: |
||||
|
||||
permissions: |
||||
contents: read |
||||
|
||||
concurrency: |
||||
group: linux-arm64-${{ github.ref }} |
||||
cancel-in-progress: true |
||||
|
||||
env: |
||||
COMPOSER_NO_INTERACTION: 1 |
||||
COMPOSER_PROCESS_TIMEOUT: 0 |
||||
|
||||
jobs: |
||||
build: |
||||
name: Build - PHP ${{ matrix.php }} ZTS |
||||
runs-on: ubuntu-22.04-arm |
||||
timeout-minutes: 90 |
||||
strategy: |
||||
fail-fast: false |
||||
matrix: |
||||
php: ["8.4", "8.5"] |
||||
env: |
||||
PHPX_HOME: ${{ github.workspace }}/vendor/swoole/phpx |
||||
|
||||
steps: |
||||
- name: Checkout TypePHP |
||||
uses: actions/checkout@v4 |
||||
|
||||
- name: Build and smoke test |
||||
uses: ./.github/actions/unix-arm64-build |
||||
with: |
||||
php-version: ${{ matrix.php }} |
||||
os: linux |
||||
library-extension: so |
||||
smoke-directory: linux-arm64 |
||||
smoke-binary: linux_arm64_smoke |
||||
smoke-output: linux-arm64-smoke-ok:zts |
||||
@ -0,0 +1,360 @@ |
||||
name: Linux x64 |
||||
|
||||
on: |
||||
push: |
||||
pull_request: |
||||
workflow_dispatch: |
||||
|
||||
permissions: |
||||
contents: read |
||||
|
||||
concurrency: |
||||
group: tests-${{ github.workflow }}-${{ github.ref }} |
||||
cancel-in-progress: true |
||||
|
||||
env: |
||||
COMPOSER_NO_INTERACTION: 1 |
||||
COMPOSER_PROCESS_TIMEOUT: 0 |
||||
|
||||
jobs: |
||||
build: |
||||
# Compilation is always required; `--skip-tests` only suppresses downstream test jobs. |
||||
name: Build - PHP ${{ matrix.php }} ZTS |
||||
runs-on: ubuntu-22.04 |
||||
timeout-minutes: 60 |
||||
strategy: |
||||
fail-fast: false |
||||
matrix: |
||||
php: ["8.4", "8.5"] |
||||
env: |
||||
PHPX_HOME: ${{ github.workspace }}/vendor/swoole/phpx |
||||
|
||||
steps: |
||||
- name: Checkout TypePHP |
||||
uses: actions/checkout@v4 |
||||
|
||||
- name: Checkout phpy |
||||
uses: actions/checkout@v4 |
||||
with: |
||||
repository: swoole/phpy |
||||
path: third_party/phpy |
||||
|
||||
- name: Setup PHP |
||||
uses: shivammathur/setup-php@v2 |
||||
with: |
||||
php-version: ${{ matrix.php }} |
||||
coverage: none |
||||
ini-values: precision=17, memory_limit=4G, error_reporting=E_ERROR|E_WARNING, display_errors=1, display_startup_errors=1, log_errors=0 |
||||
tools: composer:v2 |
||||
env: |
||||
fail-fast: true |
||||
phpts: ts |
||||
update: true |
||||
|
||||
- name: Verify ZTS PHP |
||||
shell: bash |
||||
run: | |
||||
php -r 'if (!PHP_ZTS) { fwrite(STDERR, "Expected a ZTS PHP build\n"); exit(1); }' |
||||
case "$(php -r 'echo PHP_VERSION;')" in |
||||
"${{ matrix.php }}"*) ;; |
||||
*) echo "setup-php installed an unexpected PHP version" >&2; exit 1 ;; |
||||
esac |
||||
|
||||
- name: Patch php_hash.h C++ compatibility |
||||
uses: ./.github/actions/patch-php-headers |
||||
|
||||
- name: Install native build dependencies |
||||
run: | |
||||
sudo apt-get update |
||||
sudo apt-get install --yes build-essential cmake libgmp-dev libmpfr-dev pkg-config python3-dev |
||||
|
||||
- name: Configure ZTS PHP embed library |
||||
shell: bash |
||||
run: | |
||||
php_home="$(php-config --prefix)" |
||||
embed_library="${php_home}/lib/libphp.so" |
||||
test -x "${php_home}/bin/php-config" |
||||
test -f "${embed_library}" |
||||
echo "PHP_HOME=${php_home}" >> "${GITHUB_ENV}" |
||||
echo "Using ZTS PHP $(php-config --version) embed library: ${embed_library}" |
||||
|
||||
- name: Install Composer dependencies |
||||
run: composer install --prefer-dist --no-progress |
||||
|
||||
- name: Build PHPX |
||||
shell: bash |
||||
run: | |
||||
cmake -S "${PHPX_HOME}" -B "${PHPX_HOME}/build" \ |
||||
-D CMAKE_BUILD_TYPE=Release \ |
||||
-D BUILD_TESTS=OFF \ |
||||
-D BUILD_EXT=OFF \ |
||||
-D GITHUB_ACTION=ON \ |
||||
-D php_dir="${PHP_HOME}" |
||||
cmake --build "${PHPX_HOME}/build" --target phpx --parallel 2 |
||||
test -f "${PHPX_HOME}/lib/libphpx.so" |
||||
|
||||
- name: Build phpy |
||||
working-directory: third_party/phpy |
||||
run: | |
||||
phpize |
||||
./configure |
||||
make -j2 |
||||
test -f modules/phpy.so |
||||
|
||||
- name: Enable phpy extension |
||||
shell: bash |
||||
run: | |
||||
php_ini_dir="$(php-config --ini-dir)" |
||||
echo "extension=${GITHUB_WORKSPACE}/third_party/phpy/modules/phpy.so" \ |
||||
| sudo tee "${php_ini_dir}/90-phpy.ini" |
||||
echo "PHP_INI_SCAN_DIR=${php_ini_dir}" >> "${GITHUB_ENV}" |
||||
php --ri phpy |
||||
|
||||
- name: Configure native library path |
||||
shell: bash |
||||
run: echo "LD_LIBRARY_PATH=${PHPX_HOME}/lib:${PHP_HOME}/lib" >> "${GITHUB_ENV}" |
||||
|
||||
- name: Build tpc |
||||
shell: bash |
||||
run: | |
||||
php bin/tpc.php project.yml --job 2 --no-progress |
||||
test -x ./tpc |
||||
file ./tpc |
||||
file ./tpc | grep -Eiq 'x86-64|x86_64|amd64' |
||||
./tpc --version |
||||
|
||||
- name: Upload Linux x64 build outputs |
||||
uses: actions/upload-artifact@v4 |
||||
with: |
||||
name: tpc-linux-x64-php-${{ matrix.php }}-zts |
||||
if-no-files-found: error |
||||
retention-days: 7 |
||||
path: | |
||||
tpc |
||||
vendor/swoole/phpx/lib/libphpx.so |
||||
third_party/phpy/modules/phpy.so |
||||
|
||||
phpunit: |
||||
name: PHPUnit - PHP ${{ matrix.php }} ZTS |
||||
if: ${{ startsWith(github.ref, 'refs/tags/') || !contains(github.event.head_commit.message || '', '--skip-tests') }} |
||||
runs-on: ubuntu-22.04 |
||||
timeout-minutes: 30 |
||||
needs: build |
||||
strategy: |
||||
fail-fast: false |
||||
matrix: |
||||
php: ["8.4", "8.5"] |
||||
env: |
||||
PHPX_HOME: ${{ github.workspace }}/vendor/swoole/phpx |
||||
|
||||
steps: |
||||
- name: Checkout TypePHP |
||||
uses: actions/checkout@v4 |
||||
|
||||
- name: Setup PHP |
||||
uses: shivammathur/setup-php@v2 |
||||
with: |
||||
php-version: ${{ matrix.php }} |
||||
coverage: none |
||||
extensions: curl, redis, mbstring, ffi |
||||
ini-values: ffi.enable=1, phpy.enable_operator_overloading=0, precision=17, memory_limit=4G, error_reporting=E_ERROR|E_WARNING, display_errors=1, display_startup_errors=1, log_errors=0 |
||||
tools: composer:v2 |
||||
env: |
||||
fail-fast: true |
||||
phpts: ts |
||||
update: true |
||||
|
||||
- name: Verify ZTS PHP |
||||
shell: bash |
||||
run: | |
||||
php -r 'if (!PHP_ZTS) { fwrite(STDERR, "Expected a ZTS PHP build\n"); exit(1); }' |
||||
case "$(php -r 'echo PHP_VERSION;')" in |
||||
"${{ matrix.php }}"*) ;; |
||||
*) echo "setup-php installed an unexpected PHP version" >&2; exit 1 ;; |
||||
esac |
||||
|
||||
- name: Patch php_hash.h C++ compatibility |
||||
uses: ./.github/actions/patch-php-headers |
||||
|
||||
- name: Install native build dependencies |
||||
run: | |
||||
sudo apt-get update |
||||
sudo apt-get install --yes build-essential cmake libgmp-dev libmpfr-dev pkg-config python3-dev |
||||
|
||||
- name: Install Composer dependencies |
||||
run: composer install --prefer-dist --no-progress |
||||
|
||||
- name: Download Linux x64 build outputs |
||||
uses: actions/download-artifact@v4 |
||||
with: |
||||
name: tpc-linux-x64-php-${{ matrix.php }}-zts |
||||
path: . |
||||
|
||||
- name: Verify Linux x64 build outputs |
||||
shell: bash |
||||
run: | |
||||
test -f ./tpc |
||||
test -f "${PHPX_HOME}/lib/libphpx.so" |
||||
test -f third_party/phpy/modules/phpy.so |
||||
chmod +x ./tpc |
||||
file ./tpc | grep -Eiq 'x86-64|x86_64|amd64' |
||||
|
||||
- name: Enable phpy extension |
||||
shell: bash |
||||
run: | |
||||
php_ini_dir="$(php-config --ini-dir)" |
||||
echo "extension=${GITHUB_WORKSPACE}/third_party/phpy/modules/phpy.so" \ |
||||
| sudo tee "${php_ini_dir}/90-phpy.ini" |
||||
echo "PHP_INI_SCAN_DIR=${php_ini_dir}" >> "${GITHUB_ENV}" |
||||
php --ri phpy |
||||
|
||||
- name: Configure native library path |
||||
shell: bash |
||||
run: echo "LD_LIBRARY_PATH=${PHPX_HOME}/lib:$(php-config --prefix)/lib" >> "${GITHUB_ENV}" |
||||
|
||||
- name: Run PHPUnit |
||||
run: vendor/bin/phpunit |
||||
|
||||
phpt: |
||||
name: PHPT - PHP ${{ matrix.php }} ZTS |
||||
if: ${{ startsWith(github.ref, 'refs/tags/') || !contains(github.event.head_commit.message || '', '--skip-tests') }} |
||||
runs-on: ubuntu-22.04 |
||||
timeout-minutes: 180 |
||||
needs: build |
||||
strategy: |
||||
fail-fast: false |
||||
matrix: |
||||
php: ["8.4", "8.5"] |
||||
env: |
||||
PHPX_HOME: ${{ github.workspace }}/vendor/swoole/phpx |
||||
NO_INTERACTION: 1 |
||||
REPORT_EXIT_STATUS: 1 |
||||
TYPEPHP_PHPT_GENERATED_ARTIFACT_DIR: ${{ github.workspace }}/build/phpt-generated |
||||
|
||||
steps: |
||||
- name: Checkout TypePHP |
||||
uses: actions/checkout@v4 |
||||
|
||||
- name: Setup PHP |
||||
uses: shivammathur/setup-php@v2 |
||||
with: |
||||
php-version: ${{ matrix.php }} |
||||
coverage: none |
||||
extensions: curl, redis, mbstring, ffi |
||||
ini-values: ffi.enable=1, phpy.enable_operator_overloading=0, opcache.jit=0, precision=17, memory_limit=4G, error_reporting=E_ERROR|E_WARNING, display_errors=1, display_startup_errors=1, log_errors=0 |
||||
tools: composer:v2 |
||||
env: |
||||
fail-fast: true |
||||
phpts: ts |
||||
update: true |
||||
|
||||
- name: Verify ZTS PHP |
||||
shell: bash |
||||
run: | |
||||
php -r 'if (!PHP_ZTS) { fwrite(STDERR, "Expected a ZTS PHP build\n"); exit(1); }' |
||||
case "$(php -r 'echo PHP_VERSION;')" in |
||||
"${{ matrix.php }}"*) ;; |
||||
*) echo "setup-php installed an unexpected PHP version" >&2; exit 1 ;; |
||||
esac |
||||
|
||||
- name: Patch php_hash.h C++ compatibility |
||||
uses: ./.github/actions/patch-php-headers |
||||
|
||||
- name: Install native build dependencies |
||||
run: | |
||||
sudo apt-get update |
||||
sudo apt-get install --yes build-essential cmake libgmp-dev libmpfr-dev pkg-config python3-dev |
||||
|
||||
- name: Configure ZTS PHP embed library |
||||
shell: bash |
||||
run: | |
||||
php_home="$(php-config --prefix)" |
||||
embed_library="${php_home}/lib/libphp.so" |
||||
test -x "${php_home}/bin/php-config" |
||||
test -f "${embed_library}" |
||||
echo "PHP_HOME=${php_home}" >> "${GITHUB_ENV}" |
||||
echo "Using ZTS PHP $(php-config --version) embed library: ${embed_library}" |
||||
|
||||
- name: Install Composer dependencies |
||||
run: composer install --prefer-dist --no-progress |
||||
|
||||
- name: Download Linux x64 build outputs |
||||
uses: actions/download-artifact@v4 |
||||
with: |
||||
name: tpc-linux-x64-php-${{ matrix.php }}-zts |
||||
path: . |
||||
|
||||
- name: Verify Linux x64 build outputs |
||||
shell: bash |
||||
run: | |
||||
test -f ./tpc |
||||
test -f "${PHPX_HOME}/lib/libphpx.so" |
||||
test -f third_party/phpy/modules/phpy.so |
||||
chmod +x ./tpc |
||||
file ./tpc | grep -Eiq 'x86-64|x86_64|amd64' |
||||
|
||||
- name: Enable phpy extension |
||||
shell: bash |
||||
run: | |
||||
php_ini_dir="$(php-config --ini-dir)" |
||||
echo "extension=${GITHUB_WORKSPACE}/third_party/phpy/modules/phpy.so" \ |
||||
| sudo tee "${php_ini_dir}/90-phpy.ini" |
||||
echo "PHP_INI_SCAN_DIR=${php_ini_dir}" >> "${GITHUB_ENV}" |
||||
php --ri phpy |
||||
|
||||
- name: Configure native library path |
||||
shell: bash |
||||
run: | |
||||
test -f "${PHPX_HOME}/lib/libphpx.so" |
||||
test -f "${PHP_HOME}/lib/libphp.so" |
||||
echo "LD_LIBRARY_PATH=${PHPX_HOME}/lib:${PHP_HOME}/lib" >> "${GITHUB_ENV}" |
||||
|
||||
- name: Show build environment |
||||
run: | |
||||
./tpc --version |
||||
php -v |
||||
php --ini |
||||
ldd ./tpc | grep -E 'libphp(x)?[0-9.]*\.so' |
||||
cmake --version |
||||
c++ --version |
||||
|
||||
- name: Run compiler PHPT suite with bootstrap compiler |
||||
run: | |
||||
mkdir -p build |
||||
php run-tests.php -q -j8 --compiler ./tpc \ |
||||
-w build/failed-tests.txt -W build/test-results.txt tests/compiler |
||||
|
||||
- name: Package tested Linux compiler |
||||
if: startsWith(github.ref, 'refs/tags/') && matrix.php == '8.5' |
||||
shell: bash |
||||
run: | |
||||
composer install --no-dev --prefer-dist --no-progress --classmap-authoritative |
||||
export TYPEPHP_PACKAGE_VERSION="${GITHUB_REF_NAME}" |
||||
php package.php |
||||
test "$(find . -maxdepth 1 -name 'tpc_v*_linux_*.tar.gz' -type f | wc -l)" -eq 1 |
||||
|
||||
- name: Upload Linux release package |
||||
if: startsWith(github.ref, 'refs/tags/') && matrix.php == '8.5' |
||||
uses: actions/upload-artifact@v4 |
||||
with: |
||||
name: release-linux-x64-php-${{ matrix.php }}-zts |
||||
if-no-files-found: error |
||||
retention-days: 1 |
||||
path: tpc_v*_linux_*.tar.gz |
||||
|
||||
- name: Upload PHPT failure artifacts |
||||
if: failure() |
||||
uses: actions/upload-artifact@v4 |
||||
with: |
||||
name: phpt-failures-linux-x64-php-${{ matrix.php }}-zts |
||||
if-no-files-found: ignore |
||||
retention-days: 7 |
||||
path: | |
||||
build/failed-tests.txt |
||||
build/test-results.txt |
||||
build/**/*.cc |
||||
build/**/*.h |
||||
php_test_results_*.txt |
||||
tests/compiler/**/*.diff |
||||
tests/compiler/**/*.log |
||||
tests/compiler/**/*.out |
||||
@ -0,0 +1,43 @@ |
||||
name: macOS ARM64 |
||||
|
||||
on: |
||||
push: |
||||
pull_request: |
||||
workflow_dispatch: |
||||
|
||||
permissions: |
||||
contents: read |
||||
|
||||
concurrency: |
||||
group: macos-arm64-${{ github.ref }} |
||||
cancel-in-progress: true |
||||
|
||||
env: |
||||
COMPOSER_NO_INTERACTION: 1 |
||||
COMPOSER_PROCESS_TIMEOUT: 0 |
||||
|
||||
jobs: |
||||
build: |
||||
name: Build - PHP ${{ matrix.php }} ZTS |
||||
runs-on: macos-15 |
||||
timeout-minutes: 90 |
||||
strategy: |
||||
fail-fast: false |
||||
matrix: |
||||
php: ["8.4", "8.5"] |
||||
env: |
||||
PHPX_HOME: ${{ github.workspace }}/vendor/swoole/phpx |
||||
|
||||
steps: |
||||
- name: Checkout TypePHP |
||||
uses: actions/checkout@v4 |
||||
|
||||
- name: Build and smoke test |
||||
uses: ./.github/actions/unix-arm64-build |
||||
with: |
||||
php-version: ${{ matrix.php }} |
||||
os: macos |
||||
library-extension: dylib |
||||
smoke-directory: macos |
||||
smoke-binary: macos_smoke |
||||
smoke-output: macos-arm64-smoke-ok:zts |
||||
@ -0,0 +1,141 @@ |
||||
name: Package release |
||||
|
||||
on: |
||||
push: |
||||
tags: |
||||
- "v*" |
||||
|
||||
permissions: |
||||
actions: read |
||||
contents: write |
||||
|
||||
concurrency: |
||||
group: release-${{ github.ref }} |
||||
cancel-in-progress: false |
||||
|
||||
jobs: |
||||
publish: |
||||
name: Publish ${{ github.ref_name }} |
||||
runs-on: ubuntu-22.04 |
||||
timeout-minutes: 240 |
||||
env: |
||||
GH_TOKEN: ${{ github.token }} |
||||
|
||||
steps: |
||||
- name: Wait for tested binary packages |
||||
shell: bash |
||||
run: | |
||||
set -euo pipefail |
||||
|
||||
wait_for_workflow() { |
||||
local workflow="$1" |
||||
local output_name="$2" |
||||
local run_json run_id status conclusion |
||||
|
||||
for attempt in $(seq 1 960); do |
||||
run_json="$(gh run list \ |
||||
--repo "${GITHUB_REPOSITORY}" \ |
||||
--workflow "${workflow}" \ |
||||
--commit "${GITHUB_SHA}" \ |
||||
--event push \ |
||||
--limit 20 \ |
||||
--json databaseId,status,conclusion,headBranch,createdAt)" |
||||
run_id="$(jq -r --arg tag "${GITHUB_REF_NAME}" ' |
||||
[.[] | select(.headBranch == $tag)] |
||||
| sort_by(.createdAt) |
||||
| last |
||||
| .databaseId // empty |
||||
' <<<"${run_json}")" |
||||
|
||||
if [[ -z "${run_id}" ]]; then |
||||
echo "Waiting for ${workflow} to start for tag ${GITHUB_REF_NAME}..." |
||||
sleep 15 |
||||
continue |
||||
fi |
||||
|
||||
status="$(jq -r --argjson id "${run_id}" ' |
||||
.[] | select(.databaseId == $id) | .status |
||||
' <<<"${run_json}")" |
||||
conclusion="$(jq -r --argjson id "${run_id}" ' |
||||
.[] | select(.databaseId == $id) | .conclusion |
||||
' <<<"${run_json}")" |
||||
if [[ "${status}" == "completed" ]]; then |
||||
if [[ "${conclusion}" != "success" ]]; then |
||||
echo "${workflow} run ${run_id} completed with ${conclusion}" >&2 |
||||
exit 1 |
||||
fi |
||||
echo "${workflow} run ${run_id} succeeded" |
||||
echo "${output_name}=${run_id}" >> "${GITHUB_ENV}" |
||||
return 0 |
||||
fi |
||||
|
||||
echo "Waiting for ${workflow} run ${run_id}: ${status}" |
||||
sleep 15 |
||||
done |
||||
|
||||
echo "Timed out waiting for ${workflow}" >&2 |
||||
exit 1 |
||||
} |
||||
|
||||
wait_for_workflow linux-x64.yml TESTS_RUN_ID |
||||
wait_for_workflow windows-build.yml WINDOWS_RUN_ID |
||||
wait_for_workflow linux-arm64.yml LINUX_ARM64_RUN_ID |
||||
wait_for_workflow macos-arm64.yml MACOS_ARM64_RUN_ID |
||||
|
||||
- name: Download tested packages |
||||
shell: bash |
||||
run: | |
||||
set -euo pipefail |
||||
mkdir -p dist/linux-x64 dist/linux-arm64 dist/macos-arm64 dist/windows |
||||
gh run download "${TESTS_RUN_ID}" \ |
||||
--repo "${GITHUB_REPOSITORY}" \ |
||||
--pattern 'release-linux-*' \ |
||||
--dir dist/linux-x64 |
||||
gh run download "${WINDOWS_RUN_ID}" \ |
||||
--repo "${GITHUB_REPOSITORY}" \ |
||||
--pattern 'release-windows-*' \ |
||||
--dir dist/windows |
||||
gh run download "${LINUX_ARM64_RUN_ID}" \ |
||||
--repo "${GITHUB_REPOSITORY}" \ |
||||
--pattern 'release-linux-arm64-*' \ |
||||
--dir dist/linux-arm64 |
||||
gh run download "${MACOS_ARM64_RUN_ID}" \ |
||||
--repo "${GITHUB_REPOSITORY}" \ |
||||
--pattern 'release-macos-arm64-*' \ |
||||
--dir dist/macos-arm64 |
||||
|
||||
test "$(find dist -type f -name 'tpc_v*_linux_x64.tar.gz' | wc -l)" -eq 1 |
||||
test "$(find dist -type f -name 'tpc_v*_linux_arm64.tar.gz' | wc -l)" -eq 1 |
||||
test "$(find dist -type f -name 'tpc_v*_macos_arm64.tar.gz' | wc -l)" -eq 1 |
||||
test "$(find dist -type f -name 'tpc_v*_windows_x64.zip' | wc -l)" -eq 1 |
||||
|
||||
- name: Prepare release assets |
||||
shell: bash |
||||
run: | |
||||
set -euo pipefail |
||||
mkdir -p release-assets |
||||
find dist -type f \( -name '*.tar.gz' -o -name '*.zip' \) \ |
||||
-exec cp '{}' release-assets/ \; |
||||
test "$(find release-assets -maxdepth 1 -type f \( -name '*.tar.gz' -o -name '*.zip' \) | wc -l)" -eq 4 |
||||
( |
||||
cd release-assets |
||||
sha256sum ./*.tar.gz ./*.zip > SHA256SUMS |
||||
) |
||||
ls -lh release-assets |
||||
|
||||
- name: Publish GitHub release |
||||
shell: bash |
||||
run: | |
||||
set -euo pipefail |
||||
mapfile -t assets < <(find release-assets -maxdepth 1 -type f | sort) |
||||
if gh release view "${GITHUB_REF_NAME}" --repo "${GITHUB_REPOSITORY}" >/dev/null 2>&1; then |
||||
gh release upload "${GITHUB_REF_NAME}" "${assets[@]}" \ |
||||
--repo "${GITHUB_REPOSITORY}" \ |
||||
--clobber |
||||
else |
||||
gh release create "${GITHUB_REF_NAME}" "${assets[@]}" \ |
||||
--repo "${GITHUB_REPOSITORY}" \ |
||||
--verify-tag \ |
||||
--title "TypePHP ${GITHUB_REF_NAME}" \ |
||||
--generate-notes |
||||
fi |
||||
@ -0,0 +1,449 @@ |
||||
name: Windows x64 |
||||
|
||||
on: |
||||
push: |
||||
pull_request: |
||||
workflow_dispatch: |
||||
|
||||
permissions: |
||||
contents: read |
||||
|
||||
concurrency: |
||||
group: windows-tpc-${{ github.workflow }}-${{ github.ref }} |
||||
cancel-in-progress: true |
||||
|
||||
env: |
||||
COMPOSER_NO_INTERACTION: 1 |
||||
COMPOSER_PROCESS_TIMEOUT: 0 |
||||
|
||||
jobs: |
||||
build-tpc: |
||||
name: Build - PHP ${{ matrix.php }} ZTS |
||||
runs-on: windows-2022 |
||||
timeout-minutes: 90 |
||||
strategy: |
||||
fail-fast: false |
||||
matrix: |
||||
php: ["8.4", "8.5"] |
||||
|
||||
steps: |
||||
- name: Checkout TypePHP |
||||
uses: actions/checkout@v4 |
||||
|
||||
- name: Setup PHP |
||||
uses: shivammathur/setup-php@v2 |
||||
with: |
||||
php-version: ${{ matrix.php }} |
||||
coverage: none |
||||
extensions: zip |
||||
tools: composer:v2 |
||||
env: |
||||
fail-fast: true |
||||
phpts: ts |
||||
update: true |
||||
|
||||
- name: Resolve PHP build environment |
||||
id: php-build-env |
||||
shell: pwsh |
||||
run: | |
||||
$ErrorActionPreference = 'Stop' |
||||
|
||||
$phpVersion = php -r 'echo PHP_VERSION;' |
||||
if (-not $phpVersion.StartsWith('${{ matrix.php }}.')) { |
||||
throw "setup-php installed PHP $phpVersion, expected ${{ matrix.php }}.x" |
||||
} |
||||
$threadSafety = php -r 'echo PHP_ZTS ? "zts" : "nts";' |
||||
if ($threadSafety -ne 'zts') { |
||||
throw "setup-php installed $threadSafety PHP, expected zts" |
||||
} |
||||
$zipAvailable = php -r 'echo class_exists("ZipArchive") ? "yes" : "no";' |
||||
if ($zipAvailable -ne 'yes') { |
||||
throw 'setup-php did not enable the ZipArchive extension required by package.php' |
||||
} |
||||
$phpExe = (Get-Command php).Source |
||||
$phpHome = Split-Path -Parent $phpExe |
||||
$archiveName = "php-devel-pack-$phpVersion-Win32-vs17-x64.zip" |
||||
$archive = Join-Path $env:RUNNER_TEMP $archiveName |
||||
|
||||
"PHP_VERSION=$phpVersion" | Out-File $env:GITHUB_ENV -Append -Encoding utf8 |
||||
"PHP_THREAD_SAFETY=$threadSafety" | Out-File $env:GITHUB_ENV -Append -Encoding utf8 |
||||
"PHP_HOME=$phpHome" | Out-File $env:GITHUB_ENV -Append -Encoding utf8 |
||||
"PHPX_HOME=${{ github.workspace }}\vendor\swoole\phpx" | |
||||
Out-File $env:GITHUB_ENV -Append -Encoding utf8 |
||||
"PHP_DEVEL_ARCHIVE=$archive" | Out-File $env:GITHUB_ENV -Append -Encoding utf8 |
||||
"version=$phpVersion" | Out-File $env:GITHUB_OUTPUT -Append -Encoding utf8 |
||||
"thread_safety=$threadSafety" | Out-File $env:GITHUB_OUTPUT -Append -Encoding utf8 |
||||
"archive=$archive" | Out-File $env:GITHUB_OUTPUT -Append -Encoding utf8 |
||||
|
||||
- name: Cache PHP development package |
||||
id: cache-php-devel |
||||
uses: actions/cache/restore@v4 |
||||
with: |
||||
path: ${{ steps.php-build-env.outputs.archive }} |
||||
key: windows-2022-php-devel-${{ steps.php-build-env.outputs.version }}-${{ steps.php-build-env.outputs.thread_safety }}-vs17-x64 |
||||
|
||||
- name: Install matching PHP development SDK |
||||
shell: pwsh |
||||
run: | |
||||
$ErrorActionPreference = 'Stop' |
||||
|
||||
$phpVersion = $env:PHP_VERSION |
||||
$phpHome = $env:PHP_HOME |
||||
$threadSafety = $env:PHP_THREAD_SAFETY |
||||
$archive = $env:PHP_DEVEL_ARCHIVE |
||||
$archiveName = Split-Path -Leaf $archive |
||||
$extractDir = Join-Path $env:RUNNER_TEMP "php-devel-$phpVersion" |
||||
|
||||
if (-not (Test-Path $archive)) { |
||||
$downloaded = $false |
||||
foreach ($baseUrl in @( |
||||
'https://downloads.php.net/~windows/releases', |
||||
'https://downloads.php.net/~windows/releases/archives' |
||||
)) { |
||||
try { |
||||
Invoke-WebRequest -Uri "$baseUrl/$archiveName" -OutFile $archive |
||||
$downloaded = $true |
||||
break |
||||
} catch { |
||||
Remove-Item $archive -Force -ErrorAction SilentlyContinue |
||||
} |
||||
} |
||||
if (-not $downloaded) { |
||||
throw "Unable to download the PHP $phpVersion ZTS development pack" |
||||
} |
||||
} |
||||
|
||||
Expand-Archive -Path $archive -DestinationPath $extractDir -Force |
||||
$sdkSource = Get-ChildItem $extractDir -Directory | |
||||
Where-Object { Test-Path (Join-Path $_.FullName 'include\main\php.h') } | |
||||
Select-Object -First 1 |
||||
if ($null -eq $sdkSource) { |
||||
throw "The PHP development archive has an unexpected layout: $archiveName" |
||||
} |
||||
|
||||
$sdk = Join-Path $phpHome 'SDK' |
||||
$sdkInclude = Join-Path $sdk 'include' |
||||
$sdkLib = Join-Path $sdk 'lib' |
||||
New-Item $sdkInclude, $sdkLib -ItemType Directory -Force | Out-Null |
||||
Copy-Item (Join-Path $sdkSource.FullName 'include\*') $sdkInclude -Recurse -Force |
||||
Copy-Item (Join-Path $sdkSource.FullName 'lib\*') $sdkLib -Force |
||||
|
||||
$embedLibrary = Join-Path $phpHome 'php8embed.lib' |
||||
if (-not (Test-Path $embedLibrary)) { |
||||
throw "setup-php did not install php8embed.lib in $phpHome" |
||||
} |
||||
Copy-Item $embedLibrary $sdkLib -Force |
||||
|
||||
$coreLibrary = 'php8ts.lib' |
||||
$runtimeLibrary = 'php8ts.dll' |
||||
|
||||
# Temporary compatibility fix for PHP packages predating php/php-src#22940. |
||||
$hashHeader = Join-Path $sdkInclude 'ext\hash\php_hash.h' |
||||
$hashSource = [IO.File]::ReadAllText($hashHeader) |
||||
$invalidAllocation = 'char *base = ecalloc(' |
||||
if ($hashSource.Contains($invalidAllocation)) { |
||||
$hashSource = $hashSource.Replace( |
||||
$invalidAllocation, |
||||
'char *base = (char *) ecalloc(' |
||||
) |
||||
[IO.File]::WriteAllText( |
||||
$hashHeader, |
||||
$hashSource, |
||||
[Text.UTF8Encoding]::new($false) |
||||
) |
||||
} |
||||
|
||||
foreach ($required in @( |
||||
(Join-Path $sdkInclude 'main\php.h'), |
||||
(Join-Path $sdkLib $coreLibrary), |
||||
(Join-Path $sdkLib 'php8embed.lib'), |
||||
(Join-Path $phpHome $runtimeLibrary) |
||||
)) { |
||||
if (-not (Test-Path $required)) { |
||||
throw "Required PHP SDK file is missing: $required" |
||||
} |
||||
} |
||||
|
||||
Write-Host "Using PHP $phpVersion from $phpHome" |
||||
|
||||
- name: Save PHP development package |
||||
if: steps.cache-php-devel.outputs.cache-hit != 'true' |
||||
uses: actions/cache/save@v4 |
||||
with: |
||||
path: ${{ steps.php-build-env.outputs.archive }} |
||||
key: ${{ steps.cache-php-devel.outputs.cache-primary-key }} |
||||
|
||||
- name: Install Composer dependencies |
||||
run: composer install --prefer-dist --no-progress |
||||
|
||||
- name: Patch PHPX Windows CMake target ordering |
||||
shell: pwsh |
||||
run: | |
||||
$ErrorActionPreference = 'Stop' |
||||
|
||||
$cmakeFile = Join-Path $env:PHPX_HOME 'CMakeLists.txt' |
||||
$source = [IO.File]::ReadAllText($cmakeFile).Replace("`r`n", "`n") |
||||
$copyBlockPattern = '(?ms)^ # 复制 DLL 到输出目录\n file\(GLOB MPDEC_DLLS .*?^ endforeach\(\)\n' |
||||
$copyBlock = [regex]::Match($source, $copyBlockPattern) |
||||
if (-not $copyBlock.Success) { |
||||
throw 'Unable to locate the pre-target PHPX mpdecimal copy block' |
||||
} |
||||
$source = $source.Remove($copyBlock.Index, $copyBlock.Length) |
||||
|
||||
$targetPattern = '(?ms)(add_library\(phpx SHARED \$\{SRC_FILES\}\)\nset_target_properties\(phpx PROPERTIES\n CLEAN_DIRECT_OUTPUT 1\n\)\n)' |
||||
$target = [regex]::Match($source, $targetPattern) |
||||
if (-not $target.Success) { |
||||
throw 'Unable to locate the PHPX target declaration' |
||||
} |
||||
|
||||
$copyBlockText = $copyBlock.Value.Replace(' # 复制 DLL 到输出目录', ' # Copy mpdecimal DLLs after the phpx target exists.') |
||||
$guardedCopyBlock = "`nif (IS_WINDOWS)`n$copyBlockText" + "endif()`n" |
||||
$source = $source.Insert($target.Index + $target.Length, $guardedCopyBlock) |
||||
[IO.File]::WriteAllText($cmakeFile, $source, [Text.UTF8Encoding]::new($false)) |
||||
|
||||
$targetOffset = $source.IndexOf('add_library(phpx SHARED') |
||||
$copyOffset = $source.IndexOf('add_custom_command(TARGET phpx POST_BUILD') |
||||
if ($targetOffset -lt 0 -or $copyOffset -le $targetOffset) { |
||||
throw 'PHPX post-build command still precedes its target declaration' |
||||
} |
||||
|
||||
- name: Configure MSVC |
||||
uses: ilammy/msvc-dev-cmd@v1 |
||||
with: |
||||
arch: x64 |
||||
|
||||
- name: Cache GMP and MPFR |
||||
id: cache-gmp-mpfr |
||||
uses: actions/cache/restore@v4 |
||||
with: |
||||
path: ${{ runner.temp }}\typephp-cache\vcpkg-x64-windows |
||||
key: windows-2022-msvc-vcpkg-gmp-mpfr-x64-v1 |
||||
|
||||
- name: Install GMP and MPFR |
||||
if: steps.cache-gmp-mpfr.outputs.cache-hit != 'true' |
||||
shell: pwsh |
||||
run: | |
||||
$ErrorActionPreference = 'Stop' |
||||
|
||||
$triplet = 'x64-windows' |
||||
$vcpkg = Join-Path $env:VCPKG_INSTALLATION_ROOT 'vcpkg.exe' |
||||
& $vcpkg install "gmp:$triplet" "mpfr:$triplet" |
||||
if ($LASTEXITCODE -ne 0) { |
||||
throw "vcpkg failed with exit code $LASTEXITCODE" |
||||
} |
||||
|
||||
$installed = Join-Path $env:VCPKG_INSTALLATION_ROOT "installed\$triplet" |
||||
$cache = Join-Path $env:RUNNER_TEMP 'typephp-cache\vcpkg-x64-windows' |
||||
New-Item "$cache\include", "$cache\lib", "$cache\bin" -ItemType Directory -Force | |
||||
Out-Null |
||||
Copy-Item (Join-Path $installed 'include\*') "$cache\include" -Recurse -Force |
||||
|
||||
foreach ($library in @('gmp.lib', 'gmpxx.lib', 'mpfr.lib')) { |
||||
$source = Join-Path $installed "lib\$library" |
||||
if (-not (Test-Path $source)) { |
||||
throw "vcpkg did not install $library" |
||||
} |
||||
Copy-Item $source "$cache\lib" -Force |
||||
} |
||||
Copy-Item (Join-Path $installed 'bin\*.dll') "$cache\bin" -Force |
||||
|
||||
- name: Save GMP and MPFR |
||||
if: steps.cache-gmp-mpfr.outputs.cache-hit != 'true' |
||||
uses: actions/cache/save@v4 |
||||
with: |
||||
path: ${{ runner.temp }}\typephp-cache\vcpkg-x64-windows |
||||
key: ${{ steps.cache-gmp-mpfr.outputs.cache-primary-key }} |
||||
|
||||
- name: Stage GMP and MPFR |
||||
shell: pwsh |
||||
run: | |
||||
$ErrorActionPreference = 'Stop' |
||||
|
||||
$cache = Join-Path $env:RUNNER_TEMP 'typephp-cache\vcpkg-x64-windows' |
||||
$sdkInclude = Join-Path $env:PHP_HOME 'SDK\include' |
||||
$sdkLib = Join-Path $env:PHP_HOME 'SDK\lib' |
||||
Copy-Item "$cache\include\*" $sdkInclude -Recurse -Force |
||||
|
||||
foreach ($library in @('gmp.lib', 'gmpxx.lib', 'mpfr.lib')) { |
||||
$source = Join-Path $cache "lib\$library" |
||||
if (-not (Test-Path $source)) { |
||||
throw "The dependency cache does not contain $library" |
||||
} |
||||
Copy-Item $source $sdkLib -Force |
||||
} |
||||
Copy-Item "$cache\bin\*.dll" $env:PHP_HOME -Force |
||||
|
||||
- name: Cache mpdecimal |
||||
id: cache-mpdecimal |
||||
uses: actions/cache/restore@v4 |
||||
with: |
||||
path: ${{ env.PHPX_HOME }}\thirdparty\mpdecimal\vcbuild\dist64 |
||||
key: windows-2022-msvc-mpdecimal-x64-${{ hashFiles('vendor/swoole/phpx/thirdparty/mpdecimal/**') }} |
||||
|
||||
- name: Build mpdecimal |
||||
if: steps.cache-mpdecimal.outputs.cache-hit != 'true' |
||||
shell: pwsh |
||||
run: | |
||||
$ErrorActionPreference = 'Stop' |
||||
|
||||
$buildScript = Join-Path $env:PHPX_HOME 'thirdparty\mpdecimal\vcbuild\vcbuild64.bat' |
||||
$buildDirectory = Split-Path -Parent $buildScript |
||||
Push-Location $buildDirectory |
||||
try { |
||||
& cmd.exe /d /s /c vcbuild64.bat |
||||
if ($LASTEXITCODE -ne 0) { |
||||
throw "mpdecimal build failed with exit code $LASTEXITCODE" |
||||
} |
||||
} finally { |
||||
Pop-Location |
||||
} |
||||
|
||||
- name: Save mpdecimal |
||||
if: steps.cache-mpdecimal.outputs.cache-hit != 'true' |
||||
uses: actions/cache/save@v4 |
||||
with: |
||||
path: ${{ env.PHPX_HOME }}\thirdparty\mpdecimal\vcbuild\dist64 |
||||
key: ${{ steps.cache-mpdecimal.outputs.cache-primary-key }} |
||||
|
||||
- name: Stage mpdecimal |
||||
shell: pwsh |
||||
run: | |
||||
$ErrorActionPreference = 'Stop' |
||||
|
||||
$dist = Join-Path $env:PHPX_HOME 'thirdparty\mpdecimal\vcbuild\dist64' |
||||
$sdkInclude = Join-Path $env:PHP_HOME 'SDK\include' |
||||
$sdkLib = Join-Path $env:PHP_HOME 'SDK\lib' |
||||
Copy-Item (Join-Path $dist '*.lib') $sdkLib -Force |
||||
Copy-Item (Join-Path $dist '*.dll') $sdkLib -Force |
||||
Copy-Item (Join-Path $dist '*.dll') $env:PHP_HOME -Force |
||||
Copy-Item (Join-Path $dist 'mpdecimal.h') $sdkInclude -Force |
||||
Copy-Item (Join-Path $dist 'decimal.hh') $sdkInclude -Force |
||||
|
||||
- name: Build PHPX |
||||
shell: pwsh |
||||
run: | |
||||
$ErrorActionPreference = 'Stop' |
||||
|
||||
$phpxBuild = Join-Path $env:PHPX_HOME 'build' |
||||
$sdkLib = Join-Path $env:PHP_HOME 'SDK\lib' |
||||
cmake -S $env:PHPX_HOME -B $phpxBuild -G Ninja ` |
||||
-D CMAKE_BUILD_TYPE=Release ` |
||||
-D BUILD_TESTS=OFF ` |
||||
-D BUILD_EXT=OFF ` |
||||
-D GITHUB_ACTION=ON ` |
||||
-D MPDECIMAL_LIBRARY="$sdkLib\libmpdec-4.0.1.dll.lib" ` |
||||
-D MPDECIMALXX_LIBRARY="$sdkLib\libmpdec++-4.0.1.dll.lib" |
||||
if ($LASTEXITCODE -ne 0) { |
||||
throw "PHPX configuration failed with exit code $LASTEXITCODE" |
||||
} |
||||
|
||||
cmake --build $phpxBuild --target phpx --parallel 2 |
||||
if ($LASTEXITCODE -ne 0) { |
||||
throw "PHPX build failed with exit code $LASTEXITCODE" |
||||
} |
||||
|
||||
foreach ($required in @( |
||||
(Join-Path $env:PHPX_HOME 'lib\phpx.lib'), |
||||
(Join-Path $phpxBuild 'phpx.dll') |
||||
)) { |
||||
if (-not (Test-Path $required)) { |
||||
throw "Required PHPX build output is missing: $required" |
||||
} |
||||
} |
||||
|
||||
- name: Build tpc.exe |
||||
shell: pwsh |
||||
run: | |
||||
php bin\tpc.php project.yml --job 1 --no-progress |
||||
if ($LASTEXITCODE -ne 0) { |
||||
throw "TypePHP build failed with exit code $LASTEXITCODE" |
||||
} |
||||
if (-not (Test-Path 'tpc.exe')) { |
||||
throw 'The compiler did not produce tpc.exe' |
||||
} |
||||
|
||||
- name: Run Windows smoke tests |
||||
shell: pwsh |
||||
run: | |
||||
$ErrorActionPreference = 'Stop' |
||||
|
||||
$env:PATH = "$env:PHPX_HOME\build;$env:PATH" |
||||
& .\tpc.exe tests\windows\smoke\project.yml --job 1 --no-progress |
||||
if ($LASTEXITCODE -ne 0) { |
||||
throw "Windows smoke project compilation failed with exit code $LASTEXITCODE" |
||||
} |
||||
|
||||
$smokeExe = Join-Path '${{ github.workspace }}' 'tests\windows\smoke\windows_smoke.exe' |
||||
if (-not (Test-Path $smokeExe)) { |
||||
throw "Windows smoke executable was not generated: $smokeExe" |
||||
} |
||||
|
||||
$processInfo = [Diagnostics.ProcessStartInfo]::new() |
||||
$processInfo.FileName = $smokeExe |
||||
$processInfo.ArgumentList.Add('zts') |
||||
$processInfo.UseShellExecute = $false |
||||
$processInfo.RedirectStandardOutput = $true |
||||
$processInfo.RedirectStandardError = $true |
||||
|
||||
$process = [Diagnostics.Process]::new() |
||||
$process.StartInfo = $processInfo |
||||
if (-not $process.Start()) { |
||||
throw 'Unable to start the Windows smoke executable' |
||||
} |
||||
$stdout = $process.StandardOutput.ReadToEnd() |
||||
$stderr = $process.StandardError.ReadToEnd() |
||||
$process.WaitForExit() |
||||
|
||||
Write-Host "Windows smoke stdout: $stdout" |
||||
Write-Host "Windows smoke stderr: $stderr" |
||||
Write-Host "Windows smoke exit code: $($process.ExitCode)" |
||||
if ($process.ExitCode -ne 0) { |
||||
throw "Windows smoke executable failed with exit code $($process.ExitCode)" |
||||
} |
||||
if ($stdout.Trim() -ne 'windows-smoke-ok:zts') { |
||||
throw "Unexpected Windows smoke output: $stdout" |
||||
} |
||||
|
||||
- name: Package tested Windows compiler |
||||
if: startsWith(github.ref, 'refs/tags/') && matrix.php == '8.5' |
||||
shell: pwsh |
||||
run: | |
||||
$ErrorActionPreference = 'Stop' |
||||
|
||||
composer install --no-dev --prefer-dist --no-progress --classmap-authoritative |
||||
if ($LASTEXITCODE -ne 0) { |
||||
throw "Production Composer install failed with exit code $LASTEXITCODE" |
||||
} |
||||
$env:TYPEPHP_PACKAGE_VERSION = $env:GITHUB_REF_NAME |
||||
php package.php |
||||
if ($LASTEXITCODE -ne 0) { |
||||
throw "Windows packaging failed with exit code $LASTEXITCODE" |
||||
} |
||||
|
||||
$packages = @(Get-ChildItem 'tpc_v*_windows_*.zip' -File) |
||||
if ($packages.Count -ne 1) { |
||||
throw "Expected one Windows release package, found $($packages.Count)" |
||||
} |
||||
|
||||
- name: Upload Windows release package |
||||
if: startsWith(github.ref, 'refs/tags/') && matrix.php == '8.5' |
||||
uses: actions/upload-artifact@v4 |
||||
with: |
||||
name: release-windows-x64-php-${{ matrix.php }}-zts |
||||
if-no-files-found: error |
||||
retention-days: 1 |
||||
path: tpc_v*_windows_*.zip |
||||
|
||||
- name: Upload Windows build outputs |
||||
if: always() |
||||
uses: actions/upload-artifact@v4 |
||||
with: |
||||
name: tpc-windows-x64-php-${{ matrix.php }}-zts |
||||
if-no-files-found: warn |
||||
retention-days: 7 |
||||
path: | |
||||
tpc.exe |
||||
tests/windows/smoke/windows_smoke.exe |
||||
tests/windows/smoke/build/**/*.cc |
||||
tests/windows/smoke/build/**/*.h |
||||
tests/windows/smoke/build/**/*.rsp |
||||
@ -1,103 +0,0 @@ |
||||
# CLAUDE.md |
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. |
||||
|
||||
## Project Overview |
||||
|
||||
TypePHP is a PHP native compilation project. Its `tpc` command is TypePHP Compiler (AOT), which translates PHP source code to C++, then compiles it with GCC/Clang/MSVC into native binaries. It supports Linux (primary), macOS, and Windows. |
||||
|
||||
**Prerequisites**: PHP 8.2+, GCC 9+ (C++17), CMake 3.24+. The `swoole/phpx` extension must be compiled (see README.md). |
||||
|
||||
## TypePHP Language Design Principles |
||||
|
||||
TypePHP should not blindly mirror every PHP language behavior. Most PHP syntax and semantics should remain compatible with ZendPHP, but some legal PHP constructs are historical baggage or language-design mistakes that conflict with static compilation, clear semantics, or robust generated C++ code. |
||||
|
||||
When reviewing or changing compiler behavior: |
||||
|
||||
- Prefer PHP compatibility for common, well-defined syntax that does not weaken the TypePHP static model. |
||||
- Reject PHP historical baggage when the syntax is ambiguous, surprising, or only preserved for legacy compatibility. |
||||
- Diagnose such cases as early as possible during preprocessing/static compilation, instead of deferring to runtime TypeCheck or ZendVM errors. |
||||
- Provide precise errors that include the relevant function/method name, parameter/property name, and type information where applicable. |
||||
- Compare with other statically compiled languages such as C/C++, Java, C#, Go, Rust, Kotlin, and TypeScript before deciding whether TypePHP should preserve or reject a PHP behavior. |
||||
|
||||
Example: `function test($a = 1, $b, $c) {}` is legal in PHP, but the default value for `$a` is effectively ignored and all parameters become required. This is a PHP historical compatibility artifact. TypePHP should reject it during preprocessing instead of preserving the behavior. |
||||
|
||||
Example: PHP permits `return $value;` inside `__construct()` and lets callers consume `parent::__construct()` as a value, even though constructors cannot declare a return type. TypePHP treats constructors consistently with C++/Java-style semantics: constructors initialize objects and must not return values. `return;` is allowed, but `return $value;` or using a constructor call as a value must be rejected during static compilation. |
||||
|
||||
## Build & Test Commands |
||||
|
||||
```bash |
||||
# Install PHP dependencies |
||||
composer install |
||||
|
||||
# Compile a PHP project to a native binary |
||||
./tpc <path-to-project-or-file> |
||||
|
||||
# Run all PHPUnit tests |
||||
./vendor/bin/phpunit |
||||
|
||||
# Run a single PHPUnit test class |
||||
./vendor/bin/phpunit phpunit/src/AstNodeTypeTest.php |
||||
|
||||
# Run PHPT integration tests (all) |
||||
php run-tests.php tests/compiler/ |
||||
|
||||
# Run a single PHPT test |
||||
php run-tests.php tests/compiler/arrays.phpt |
||||
``` |
||||
|
||||
## Architecture |
||||
|
||||
### Translation Pipeline |
||||
|
||||
The compiler follows a 4-stage pipeline, orchestrated by `src/Translator.php` (the main entry point): |
||||
|
||||
1. **prepare()** — Scan PHP files, collect symbol declarations and dependencies, topological-sort for compilation order |
||||
2. **convert()** — Parse PHP AST via `nikic/php-parser`, translate each node to C++ source code |
||||
3. **compile()** — Invoke the platform C++ compiler (GCC/Clang/MSVC) on generated `.cc` files |
||||
4. **build()** — Link object files into a native binary executable |
||||
|
||||
### Class Hierarchy |
||||
|
||||
``` |
||||
src/CompilerBase.php (core PHP→C++ translation logic, indent/output/mode helpers) |
||||
├─ uses traits: AstNodeType, FuncCallOptimizer, ClosureGenerator, |
||||
│ PlaceHolderGenerator, PropertyPromotion, MagicMethodDetector |
||||
└─ src/Preprocessor.php (scanning, symbol tables, dependency sort, YAML config) |
||||
└─ src/Translator.php (full pipeline: prepare→convert→compile→build) |
||||
└─ src/CompilerTest.php (test-only subclass, used by PHPUnit tests) |
||||
``` |
||||
|
||||
### Key Components |
||||
|
||||
| Directory | Purpose | |
||||
|-----------|---------| |
||||
| `src/Entity/` | Data classes: `ClassDef`, `FunctionDef`, `MethodDef`, `PropertyDef`, `ConstantDef`, `InterfaceDef` | |
||||
| `src/Generator/` | Codegen helpers: `ClosureGenerator`, `PlaceHolderGenerator`, `PropertyPromotion`, `Utils` | |
||||
| `src/Backend/` | Compiler abstraction: `CompilerBackend` (abstract) → `Gcc`, `Clang`, `Msvc`. Factory pattern via `CompilerFactory` | |
||||
| `src/Platform/` | OS abstraction: `PlatformBase` → `Linux`, `Macos`, `Windows`. Factory via `PlatformFactory` | |
||||
| `src/Context/` | `ScopeContext` and `FunctionContext` for variable scoping and type tracking | |
||||
| `src/Exception/` | `SyntaxError`, `Unsupported`, `DynamicCall`, `PlaceHolder`, `Skip`, `Redo`, `TestError` | |
||||
| `src/Parser/` | Special-purpose parsers like `StdContainerParser` (C++ std container foreach support) | |
||||
| `src/Resolver/Reflection.php` | Static helpers wrapping PHP reflection (internal class/function detection) | |
||||
| `src/Generator/Symbol.php` | Maps PHP operations to `phpx` C++ API symbol names | |
||||
| `src/Build/FileScanner.php` | Recursive file discovery with extension filtering (supports `.php`, `.cpp`, `.c`, `.s`, `.m`, `.mm`) | |
||||
| `src/Entity/ArgInfo.php` | Generates C function argument info structures for internal function registration | |
||||
| `src/Extractor.php` | Extracts interfaces from PHP classes | |
||||
| `src/Transform/Visitor.php` | Base `NodeVisitorAbstract` extension (skeleton for custom AST visitors) | |
||||
|
||||
### Configuration |
||||
|
||||
- `project.yml` — per-project build config (name, build-mode, C++ standard, compiler flags, sources, resources/icon) |
||||
- Command-line arguments and YAML config are merged in `Preprocessor`, with CLI taking highest priority |
||||
|
||||
### Generated Output |
||||
|
||||
Generated `.cc` and `.o` files land in `build/` directory. The compiled binary is named from `project.yml`'s `name` field (default: `app`). |
||||
|
||||
### Test Infrastructure |
||||
|
||||
- **PHPUnit tests** (`phpunit/src/`) — unit/integration tests for compiler internals. Bootstrap at `phpunit/bootstrap.php` defines a `BaseTest` class with an `exec()` helper that runs the compiler and expects a `TestError` exception containing a given string |
||||
- **PHPT tests** (`tests/compiler/`) — end-to-end tests using the standard PHPT format (`run-tests.php`). Each `.phpt` contains PHP source and expected output sections |
||||
|
||||
When writing new compiler tests, use `CompilerTest::create(ROOT_PATH)` (in `src/CompilerTest.php`) which sets `forTest = true` to enable test-specific behavior without writing files to disk. |
||||
@ -0,0 +1,674 @@ |
||||
GNU GENERAL PUBLIC LICENSE |
||||
Version 3, 29 June 2007 |
||||
|
||||
Copyright (C) 2026 上海识沃网络科技有限公司. <https://www.swoole.com/> |
||||
Everyone is permitted to copy and distribute verbatim copies |
||||
of this license document, but changing it is not allowed. |
||||
|
||||
Preamble |
||||
|
||||
The GNU General Public License is a free, copyleft license for |
||||
software and other kinds of works. |
||||
|
||||
The licenses for most software and other practical works are designed |
||||
to take away your freedom to share and change the works. By contrast, |
||||
the GNU General Public License is intended to guarantee your freedom to |
||||
share and change all versions of a program--to make sure it remains free |
||||
software for all its users. We, the Free Software Foundation, use the |
||||
GNU General Public License for most of our software; it applies also to |
||||
any other work released this way by its authors. You can apply it to |
||||
your programs, too. |
||||
|
||||
When we speak of free software, we are referring to freedom, not |
||||
price. Our General Public Licenses are designed to make sure that you |
||||
have the freedom to distribute copies of free software (and charge for |
||||
them if you wish), that you receive source code or can get it if you |
||||
want it, that you can change the software or use pieces of it in new |
||||
free programs, and that you know you can do these things. |
||||
|
||||
To protect your rights, we need to prevent others from denying you |
||||
these rights or asking you to surrender the rights. Therefore, you have |
||||
certain responsibilities if you distribute copies of the software, or if |
||||
you modify it: responsibilities to respect the freedom of others. |
||||
|
||||
For example, if you distribute copies of such a program, whether |
||||
gratis or for a fee, you must pass on to the recipients the same |
||||
freedoms that you received. You must make sure that they, too, receive |
||||
or can get the source code. And you must show them these terms so they |
||||
know their rights. |
||||
|
||||
Developers that use the GNU GPL protect your rights with two steps: |
||||
(1) assert copyright on the software, and (2) offer you this License |
||||
giving you legal permission to copy, distribute and/or modify it. |
||||
|
||||
For the developers' and authors' protection, the GPL clearly explains |
||||
that there is no warranty for this free software. For both users' and |
||||
authors' sake, the GPL requires that modified versions be marked as |
||||
changed, so that their problems will not be attributed erroneously to |
||||
authors of previous versions. |
||||
|
||||
Some devices are designed to deny users access to install or run |
||||
modified versions of the software inside them, although the manufacturer |
||||
can do so. This is fundamentally incompatible with the aim of |
||||
protecting users' freedom to change the software. The systematic |
||||
pattern of such abuse occurs in the area of products for individuals to |
||||
use, which is precisely where it is most unacceptable. Therefore, we |
||||
have designed this version of the GPL to prohibit the practice for those |
||||
products. If such problems arise substantially in other domains, we |
||||
stand ready to extend this provision to those domains in future versions |
||||
of the GPL, as needed to protect the freedom of users. |
||||
|
||||
Finally, every program is threatened constantly by software patents. |
||||
States should not allow patents to restrict development and use of |
||||
software on general-purpose computers, but in those that do, we wish to |
||||
avoid the special danger that patents applied to a free program could |
||||
make it effectively proprietary. To prevent this, the GPL assures that |
||||
patents cannot be used to render the program non-free. |
||||
|
||||
The precise terms and conditions for copying, distribution and |
||||
modification follow. |
||||
|
||||
TERMS AND CONDITIONS |
||||
|
||||
0. Definitions. |
||||
|
||||
"This License" refers to version 3 of the GNU General Public License. |
||||
|
||||
"Copyright" also means copyright-like laws that apply to other kinds of |
||||
works, such as semiconductor masks. |
||||
|
||||
"The Program" refers to any copyrightable work licensed under this |
||||
License. Each licensee is addressed as "you". "Licensees" and |
||||
"recipients" may be individuals or organizations. |
||||
|
||||
To "modify" a work means to copy from or adapt all or part of the work |
||||
in a fashion requiring copyright permission, other than the making of an |
||||
exact copy. The resulting work is called a "modified version" of the |
||||
earlier work or a work "based on" the earlier work. |
||||
|
||||
A "covered work" means either the unmodified Program or a work based |
||||
on the Program. |
||||
|
||||
To "propagate" a work means to do anything with it that, without |
||||
permission, would make you directly or secondarily liable for |
||||
infringement under applicable copyright law, except executing it on a |
||||
computer or modifying a private copy. Propagation includes copying, |
||||
distribution (with or without modification), making available to the |
||||
public, and in some countries other activities as well. |
||||
|
||||
To "convey" a work means any kind of propagation that enables other |
||||
parties to make or receive copies. Mere interaction with a user through |
||||
a computer network, with no transfer of a copy, is not conveying. |
||||
|
||||
An interactive user interface displays "Appropriate Legal Notices" |
||||
to the extent that it includes a convenient and prominently visible |
||||
feature that (1) displays an appropriate copyright notice, and (2) |
||||
tells the user that there is no warranty for the work (except to the |
||||
extent that warranties are provided), that licensees may convey the |
||||
work under this License, and how to view a copy of this License. If |
||||
the interface presents a list of user commands or options, such as a |
||||
menu, a prominent item in the list meets this criterion. |
||||
|
||||
1. Source Code. |
||||
|
||||
The "source code" for a work means the preferred form of the work |
||||
for making modifications to it. "Object code" means any non-source |
||||
form of a work. |
||||
|
||||
A "Standard Interface" means an interface that either is an official |
||||
standard defined by a recognized standards body, or, in the case of |
||||
interfaces specified for a particular programming language, one that |
||||
is widely used among developers working in that language. |
||||
|
||||
The "System Libraries" of an executable work include anything, other |
||||
than the work as a whole, that (a) is included in the normal form of |
||||
packaging a Major Component, but which is not part of that Major |
||||
Component, and (b) serves only to enable use of the work with that |
||||
Major Component, or to implement a Standard Interface for which an |
||||
implementation is available to the public in source code form. A |
||||
"Major Component", in this context, means a major essential component |
||||
(kernel, window system, and so on) of the specific operating system |
||||
(if any) on which the executable work runs, or a compiler used to |
||||
produce the work, or an object code interpreter used to run it. |
||||
|
||||
The "Corresponding Source" for a work in object code form means all |
||||
the source code needed to generate, install, and (for an executable |
||||
work) run the object code and to modify the work, including scripts to |
||||
control those activities. However, it does not include the work's |
||||
System Libraries, or general-purpose tools or generally available free |
||||
programs which are used unmodified in performing those activities but |
||||
which are not part of the work. For example, Corresponding Source |
||||
includes interface definition files associated with source files for |
||||
the work, and the source code for shared libraries and dynamically |
||||
linked subprograms that the work is specifically designed to require, |
||||
such as by intimate data communication or control flow between those |
||||
subprograms and other parts of the work. |
||||
|
||||
The Corresponding Source need not include anything that users |
||||
can regenerate automatically from other parts of the Corresponding |
||||
Source. |
||||
|
||||
The Corresponding Source for a work in source code form is that |
||||
same work. |
||||
|
||||
2. Basic Permissions. |
||||
|
||||
All rights granted under this License are granted for the term of |
||||
copyright on the Program, and are irrevocable provided the stated |
||||
conditions are met. This License explicitly affirms your unlimited |
||||
permission to run the unmodified Program. The output from running a |
||||
covered work is covered by this License only if the output, given its |
||||
content, constitutes a covered work. This License acknowledges your |
||||
rights of fair use or other equivalent, as provided by copyright law. |
||||
|
||||
You may make, run and propagate covered works that you do not |
||||
convey, without conditions so long as your license otherwise remains |
||||
in force. You may convey covered works to others for the sole purpose |
||||
of having them make modifications exclusively for you, or provide you |
||||
with facilities for running those works, provided that you comply with |
||||
the terms of this License in conveying all material for which you do |
||||
not control copyright. Those thus making or running the covered works |
||||
for you must do so exclusively on your behalf, under your direction |
||||
and control, on terms that prohibit them from making any copies of |
||||
your copyrighted material outside their relationship with you. |
||||
|
||||
Conveying under any other circumstances is permitted solely under |
||||
the conditions stated below. Sublicensing is not allowed; section 10 |
||||
makes it unnecessary. |
||||
|
||||
3. Protecting Users' Legal Rights From Anti-Circumvention Law. |
||||
|
||||
No covered work shall be deemed part of an effective technological |
||||
measure under any applicable law fulfilling obligations under article |
||||
11 of the WIPO copyright treaty adopted on 20 December 1996, or |
||||
similar laws prohibiting or restricting circumvention of such |
||||
measures. |
||||
|
||||
When you convey a covered work, you waive any legal power to forbid |
||||
circumvention of technological measures to the extent such circumvention |
||||
is effected by exercising rights under this License with respect to |
||||
the covered work, and you disclaim any intention to limit operation or |
||||
modification of the work as a means of enforcing, against the work's |
||||
users, your or third parties' legal rights to forbid circumvention of |
||||
technological measures. |
||||
|
||||
4. Conveying Verbatim Copies. |
||||
|
||||
You may convey verbatim copies of the Program's source code as you |
||||
receive it, in any medium, provided that you conspicuously and |
||||
appropriately publish on each copy an appropriate copyright notice; |
||||
keep intact all notices stating that this License and any |
||||
non-permissive terms added in accord with section 7 apply to the code; |
||||
keep intact all notices of the absence of any warranty; and give all |
||||
recipients a copy of this License along with the Program. |
||||
|
||||
You may charge any price or no price for each copy that you convey, |
||||
and you may offer support or warranty protection for a fee. |
||||
|
||||
5. Conveying Modified Source Versions. |
||||
|
||||
You may convey a work based on the Program, or the modifications to |
||||
produce it from the Program, in the form of source code under the |
||||
terms of section 4, provided that you also meet all of these conditions: |
||||
|
||||
a) The work must carry prominent notices stating that you modified |
||||
it, and giving a relevant date. |
||||
|
||||
b) The work must carry prominent notices stating that it is |
||||
released under this License and any conditions added under section |
||||
7. This requirement modifies the requirement in section 4 to |
||||
"keep intact all notices". |
||||
|
||||
c) You must license the entire work, as a whole, under this |
||||
License to anyone who comes into possession of a copy. This |
||||
License will therefore apply, along with any applicable section 7 |
||||
additional terms, to the whole of the work, and all its parts, |
||||
regardless of how they are packaged. This License gives no |
||||
permission to license the work in any other way, but it does not |
||||
invalidate such permission if you have separately received it. |
||||
|
||||
d) If the work has interactive user interfaces, each must display |
||||
Appropriate Legal Notices; however, if the Program has interactive |
||||
interfaces that do not display Appropriate Legal Notices, your |
||||
work need not make them do so. |
||||
|
||||
A compilation of a covered work with other separate and independent |
||||
works, which are not by their nature extensions of the covered work, |
||||
and which are not combined with it such as to form a larger program, |
||||
in or on a volume of a storage or distribution medium, is called an |
||||
"aggregate" if the compilation and its resulting copyright are not |
||||
used to limit the access or legal rights of the compilation's users |
||||
beyond what the individual works permit. Inclusion of a covered work |
||||
in an aggregate does not cause this License to apply to the other |
||||
parts of the aggregate. |
||||
|
||||
6. Conveying Non-Source Forms. |
||||
|
||||
You may convey a covered work in object code form under the terms |
||||
of sections 4 and 5, provided that you also convey the |
||||
machine-readable Corresponding Source under the terms of this License, |
||||
in one of these ways: |
||||
|
||||
a) Convey the object code in, or embodied in, a physical product |
||||
(including a physical distribution medium), accompanied by the |
||||
Corresponding Source fixed on a durable physical medium |
||||
customarily used for software interchange. |
||||
|
||||
b) Convey the object code in, or embodied in, a physical product |
||||
(including a physical distribution medium), accompanied by a |
||||
written offer, valid for at least three years and valid for as |
||||
long as you offer spare parts or customer support for that product |
||||
model, to give anyone who possesses the object code either (1) a |
||||
copy of the Corresponding Source for all the software in the |
||||
product that is covered by this License, on a durable physical |
||||
medium customarily used for software interchange, for a price no |
||||
more than your reasonable cost of physically performing this |
||||
conveying of source, or (2) access to copy the |
||||
Corresponding Source from a network server at no charge. |
||||
|
||||
c) Convey individual copies of the object code with a copy of the |
||||
written offer to provide the Corresponding Source. This |
||||
alternative is allowed only occasionally and noncommercially, and |
||||
only if you received the object code with such an offer, in accord |
||||
with subsection 6b. |
||||
|
||||
d) Convey the object code by offering access from a designated |
||||
place (gratis or for a charge), and offer equivalent access to the |
||||
Corresponding Source in the same way through the same place at no |
||||
further charge. You need not require recipients to copy the |
||||
Corresponding Source along with the object code. If the place to |
||||
copy the object code is a network server, the Corresponding Source |
||||
may be on a different server (operated by you or a third party) |
||||
that supports equivalent copying facilities, provided you maintain |
||||
clear directions next to the object code saying where to find the |
||||
Corresponding Source. Regardless of what server hosts the |
||||
Corresponding Source, you remain obligated to ensure that it is |
||||
available for as long as needed to satisfy these requirements. |
||||
|
||||
e) Convey the object code using peer-to-peer transmission, provided |
||||
you inform other peers where the object code and Corresponding |
||||
Source of the work are being offered to the general public at no |
||||
charge under subsection 6d. |
||||
|
||||
A separable portion of the object code, whose source code is excluded |
||||
from the Corresponding Source as a System Library, need not be |
||||
included in conveying the object code work. |
||||
|
||||
A "User Product" is either (1) a "consumer product", which means any |
||||
tangible personal property which is normally used for personal, family, |
||||
or household purposes, or (2) anything designed or sold for incorporation |
||||
into a dwelling. In determining whether a product is a consumer product, |
||||
doubtful cases shall be resolved in favor of coverage. For a particular |
||||
product received by a particular user, "normally used" refers to a |
||||
typical or common use of that class of product, regardless of the status |
||||
of the particular user or of the way in which the particular user |
||||
actually uses, or expects or is expected to use, the product. A product |
||||
is a consumer product regardless of whether the product has substantial |
||||
commercial, industrial or non-consumer uses, unless such uses represent |
||||
the only significant mode of use of the product. |
||||
|
||||
"Installation Information" for a User Product means any methods, |
||||
procedures, authorization keys, or other information required to install |
||||
and execute modified versions of a covered work in that User Product from |
||||
a modified version of its Corresponding Source. The information must |
||||
suffice to ensure that the continued functioning of the modified object |
||||
code is in no case prevented or interfered with solely because |
||||
modification has been made. |
||||
|
||||
If you convey an object code work under this section in, or with, or |
||||
specifically for use in, a User Product, and the conveying occurs as |
||||
part of a transaction in which the right of possession and use of the |
||||
User Product is transferred to the recipient in perpetuity or for a |
||||
fixed term (regardless of how the transaction is characterized), the |
||||
Corresponding Source conveyed under this section must be accompanied |
||||
by the Installation Information. But this requirement does not apply |
||||
if neither you nor any third party retains the ability to install |
||||
modified object code on the User Product (for example, the work has |
||||
been installed in ROM). |
||||
|
||||
The requirement to provide Installation Information does not include a |
||||
requirement to continue to provide support service, warranty, or updates |
||||
for a work that has been modified or installed by the recipient, or for |
||||
the User Product in which it has been modified or installed. Access to a |
||||
network may be denied when the modification itself materially and |
||||
adversely affects the operation of the network or violates the rules and |
||||
protocols for communication across the network. |
||||
|
||||
Corresponding Source conveyed, and Installation Information provided, |
||||
in accord with this section must be in a format that is publicly |
||||
documented (and with an implementation available to the public in |
||||
source code form), and must require no special password or key for |
||||
unpacking, reading or copying. |
||||
|
||||
7. Additional Terms. |
||||
|
||||
"Additional permissions" are terms that supplement the terms of this |
||||
License by making exceptions from one or more of its conditions. |
||||
Additional permissions that are applicable to the entire Program shall |
||||
be treated as though they were included in this License, to the extent |
||||
that they are valid under applicable law. If additional permissions |
||||
apply only to part of the Program, that part may be used separately |
||||
under those permissions, but the entire Program remains governed by |
||||
this License without regard to the additional permissions. |
||||
|
||||
When you convey a copy of a covered work, you may at your option |
||||
remove any additional permissions from that copy, or from any part of |
||||
it. (Additional permissions may be written to require their own |
||||
removal in certain cases when you modify the work.) You may place |
||||
additional permissions on material, added by you to a covered work, |
||||
for which you have or can give appropriate copyright permission. |
||||
|
||||
Notwithstanding any other provision of this License, for material you |
||||
add to a covered work, you may (if authorized by the copyright holders of |
||||
that material) supplement the terms of this License with terms: |
||||
|
||||
a) Disclaiming warranty or limiting liability differently from the |
||||
terms of sections 15 and 16 of this License; or |
||||
|
||||
b) Requiring preservation of specified reasonable legal notices or |
||||
author attributions in that material or in the Appropriate Legal |
||||
Notices displayed by works containing it; or |
||||
|
||||
c) Prohibiting misrepresentation of the origin of that material, or |
||||
requiring that modified versions of such material be marked in |
||||
reasonable ways as different from the original version; or |
||||
|
||||
d) Limiting the use for publicity purposes of names of licensors or |
||||
authors of the material; or |
||||
|
||||
e) Declining to grant rights under trademark law for use of some |
||||
trade names, trademarks, or service marks; or |
||||
|
||||
f) Requiring indemnification of licensors and authors of that |
||||
material by anyone who conveys the material (or modified versions of |
||||
it) with contractual assumptions of liability to the recipient, for |
||||
any liability that these contractual assumptions directly impose on |
||||
those licensors and authors. |
||||
|
||||
All other non-permissive additional terms are considered "further |
||||
restrictions" within the meaning of section 10. If the Program as you |
||||
received it, or any part of it, contains a notice stating that it is |
||||
governed by this License along with a term that is a further |
||||
restriction, you may remove that term. If a license document contains |
||||
a further restriction but permits relicensing or conveying under this |
||||
License, you may add to a covered work material governed by the terms |
||||
of that license document, provided that the further restriction does |
||||
not survive such relicensing or conveying. |
||||
|
||||
If you add terms to a covered work in accord with this section, you |
||||
must place, in the relevant source files, a statement of the |
||||
additional terms that apply to those files, or a notice indicating |
||||
where to find the applicable terms. |
||||
|
||||
Additional terms, permissive or non-permissive, may be stated in the |
||||
form of a separately written license, or stated as exceptions; |
||||
the above requirements apply either way. |
||||
|
||||
8. Termination. |
||||
|
||||
You may not propagate or modify a covered work except as expressly |
||||
provided under this License. Any attempt otherwise to propagate or |
||||
modify it is void, and will automatically terminate your rights under |
||||
this License (including any patent licenses granted under the third |
||||
paragraph of section 11). |
||||
|
||||
However, if you cease all violation of this License, then your |
||||
license from a particular copyright holder is reinstated (a) |
||||
provisionally, unless and until the copyright holder explicitly and |
||||
finally terminates your license, and (b) permanently, if the copyright |
||||
holder fails to notify you of the violation by some reasonable means |
||||
prior to 60 days after the cessation. |
||||
|
||||
Moreover, your license from a particular copyright holder is |
||||
reinstated permanently if the copyright holder notifies you of the |
||||
violation by some reasonable means, this is the first time you have |
||||
received notice of violation of this License (for any work) from that |
||||
copyright holder, and you cure the violation prior to 30 days after |
||||
your receipt of the notice. |
||||
|
||||
Termination of your rights under this section does not terminate the |
||||
licenses of parties who have received copies or rights from you under |
||||
this License. If your rights have been terminated and not permanently |
||||
reinstated, you do not qualify to receive new licenses for the same |
||||
material under section 10. |
||||
|
||||
9. Acceptance Not Required for Having Copies. |
||||
|
||||
You are not required to accept this License in order to receive or |
||||
run a copy of the Program. Ancillary propagation of a covered work |
||||
occurring solely as a consequence of using peer-to-peer transmission |
||||
to receive a copy likewise does not require acceptance. However, |
||||
nothing other than this License grants you permission to propagate or |
||||
modify any covered work. These actions infringe copyright if you do |
||||
not accept this License. Therefore, by modifying or propagating a |
||||
covered work, you indicate your acceptance of this License to do so. |
||||
|
||||
10. Automatic Licensing of Downstream Recipients. |
||||
|
||||
Each time you convey a covered work, the recipient automatically |
||||
receives a license from the original licensors, to run, modify and |
||||
propagate that work, subject to this License. You are not responsible |
||||
for enforcing compliance by third parties with this License. |
||||
|
||||
An "entity transaction" is a transaction transferring control of an |
||||
organization, or substantially all assets of one, or subdividing an |
||||
organization, or merging organizations. If propagation of a covered |
||||
work results from an entity transaction, each party to that |
||||
transaction who receives a copy of the work also receives whatever |
||||
licenses to the work the party's predecessor in interest had or could |
||||
give under the previous paragraph, plus a right to possession of the |
||||
Corresponding Source of the work from the predecessor in interest, if |
||||
the predecessor has it or can get it with reasonable efforts. |
||||
|
||||
You may not impose any further restrictions on the exercise of the |
||||
rights granted or affirmed under this License. For example, you may |
||||
not impose a license fee, royalty, or other charge for exercise of |
||||
rights granted under this License, and you may not initiate litigation |
||||
(including a cross-claim or counterclaim in a lawsuit) alleging that |
||||
any patent claim is infringed by making, using, selling, offering for |
||||
sale, or importing the Program or any portion of it. |
||||
|
||||
11. Patents. |
||||
|
||||
A "contributor" is a copyright holder who authorizes use under this |
||||
License of the Program or a work on which the Program is based. The |
||||
work thus licensed is called the contributor's "contributor version". |
||||
|
||||
A contributor's "essential patent claims" are all patent claims |
||||
owned or controlled by the contributor, whether already acquired or |
||||
hereafter acquired, that would be infringed by some manner, permitted |
||||
by this License, of making, using, or selling its contributor version, |
||||
but do not include claims that would be infringed only as a |
||||
consequence of further modification of the contributor version. For |
||||
purposes of this definition, "control" includes the right to grant |
||||
patent sublicenses in a manner consistent with the requirements of |
||||
this License. |
||||
|
||||
Each contributor grants you a non-exclusive, worldwide, royalty-free |
||||
patent license under the contributor's essential patent claims, to |
||||
make, use, sell, offer for sale, import and otherwise run, modify and |
||||
propagate the contents of its contributor version. |
||||
|
||||
In the following three paragraphs, a "patent license" is any express |
||||
agreement or commitment, however denominated, not to enforce a patent |
||||
(such as an express permission to practice a patent or covenant not to |
||||
sue for patent infringement). To "grant" such a patent license to a |
||||
party means to make such an agreement or commitment not to enforce a |
||||
patent against the party. |
||||
|
||||
If you convey a covered work, knowingly relying on a patent license, |
||||
and the Corresponding Source of the work is not available for anyone |
||||
to copy, free of charge and under the terms of this License, through a |
||||
publicly available network server or other readily accessible means, |
||||
then you must either (1) cause the Corresponding Source to be so |
||||
available, or (2) arrange to deprive yourself of the benefit of the |
||||
patent license for this particular work, or (3) arrange, in a manner |
||||
consistent with the requirements of this License, to extend the patent |
||||
license to downstream recipients. "Knowingly relying" means you have |
||||
actual knowledge that, but for the patent license, your conveying the |
||||
covered work in a country, or your recipient's use of the covered work |
||||
in a country, would infringe one or more identifiable patents in that |
||||
country that you have reason to believe are valid. |
||||
|
||||
If, pursuant to or in connection with a single transaction or |
||||
arrangement, you convey, or propagate by procuring conveyance of, a |
||||
covered work, and grant a patent license to some of the parties |
||||
receiving the covered work authorizing them to use, propagate, modify |
||||
or convey a specific copy of the covered work, then the patent license |
||||
you grant is automatically extended to all recipients of the covered |
||||
work and works based on it. |
||||
|
||||
A patent license is "discriminatory" if it does not include within |
||||
the scope of its coverage, prohibits the exercise of, or is |
||||
conditioned on the non-exercise of one or more of the rights that are |
||||
specifically granted under this License. You may not convey a covered |
||||
work if you are a party to an arrangement with a third party that is |
||||
in the business of distributing software, under which you make payment |
||||
to the third party based on the extent of your activity of conveying |
||||
the work, and under which the third party grants, to any of the |
||||
parties who would receive the covered work from you, a discriminatory |
||||
patent license (a) in connection with copies of the covered work |
||||
conveyed by you (or copies made from those copies), or (b) primarily |
||||
for and in connection with specific products or compilations that |
||||
contain the covered work, unless you entered into that arrangement, |
||||
or that patent license was granted, prior to 28 March 2007. |
||||
|
||||
Nothing in this License shall be construed as excluding or limiting |
||||
any implied license or other defenses to infringement that may |
||||
otherwise be available to you under applicable patent law. |
||||
|
||||
12. No Surrender of Others' Freedom. |
||||
|
||||
If conditions are imposed on you (whether by court order, agreement or |
||||
otherwise) that contradict the conditions of this License, they do not |
||||
excuse you from the conditions of this License. If you cannot convey a |
||||
covered work so as to satisfy simultaneously your obligations under this |
||||
License and any other pertinent obligations, then as a consequence you may |
||||
not convey it at all. For example, if you agree to terms that obligate you |
||||
to collect a royalty for further conveying from those to whom you convey |
||||
the Program, the only way you could satisfy both those terms and this |
||||
License would be to refrain entirely from conveying the Program. |
||||
|
||||
13. Use with the GNU Affero General Public License. |
||||
|
||||
Notwithstanding any other provision of this License, you have |
||||
permission to link or combine any covered work with a work licensed |
||||
under version 3 of the GNU Affero General Public License into a single |
||||
combined work, and to convey the resulting work. The terms of this |
||||
License will continue to apply to the part which is the covered work, |
||||
but the special requirements of the GNU Affero General Public License, |
||||
section 13, concerning interaction through a network will apply to the |
||||
combination as such. |
||||
|
||||
14. Revised Versions of this License. |
||||
|
||||
The Free Software Foundation may publish revised and/or new versions of |
||||
the GNU General Public License from time to time. Such new versions will |
||||
be similar in spirit to the present version, but may differ in detail to |
||||
address new problems or concerns. |
||||
|
||||
Each version is given a distinguishing version number. If the |
||||
Program specifies that a certain numbered version of the GNU General |
||||
Public License "or any later version" applies to it, you have the |
||||
option of following the terms and conditions either of that numbered |
||||
version or of any later version published by the Free Software |
||||
Foundation. If the Program does not specify a version number of the |
||||
GNU General Public License, you may choose any version ever published |
||||
by the Free Software Foundation. |
||||
|
||||
If the Program specifies that a proxy can decide which future |
||||
versions of the GNU General Public License can be used, that proxy's |
||||
public statement of acceptance of a version permanently authorizes you |
||||
to choose that version for the Program. |
||||
|
||||
Later license versions may give you additional or different |
||||
permissions. However, no additional obligations are imposed on any |
||||
author or copyright holder as a result of your choosing to follow a |
||||
later version. |
||||
|
||||
15. Disclaimer of Warranty. |
||||
|
||||
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY |
||||
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT |
||||
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY |
||||
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, |
||||
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR |
||||
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM |
||||
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF |
||||
ALL NECESSARY SERVICING, REPAIR OR CORRECTION. |
||||
|
||||
16. Limitation of Liability. |
||||
|
||||
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING |
||||
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS |
||||
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY |
||||
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE |
||||
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF |
||||
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD |
||||
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), |
||||
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF |
||||
SUCH DAMAGES. |
||||
|
||||
17. Interpretation of Sections 15 and 16. |
||||
|
||||
If the disclaimer of warranty and limitation of liability provided |
||||
above cannot be given local legal effect according to their terms, |
||||
reviewing courts shall apply local law that most closely approximates |
||||
an absolute waiver of all civil liability in connection with the |
||||
Program, unless a warranty or assumption of liability accompanies a |
||||
copy of the Program in return for a fee. |
||||
|
||||
END OF TERMS AND CONDITIONS |
||||
|
||||
How to Apply These Terms to Your New Programs |
||||
|
||||
If you develop a new program, and you want it to be of the greatest |
||||
possible use to the public, the best way to achieve this is to make it |
||||
free software which everyone can redistribute and change under these terms. |
||||
|
||||
To do so, attach the following notices to the program. It is safest |
||||
to attach them to the start of each source file to most effectively |
||||
state the exclusion of warranty; and each file should have at least |
||||
the "copyright" line and a pointer to where the full notice is found. |
||||
|
||||
<one line to give the program's name and a brief idea of what it does.> |
||||
Copyright (C) <year> <name of author> |
||||
|
||||
This program is free software: you can redistribute it and/or modify |
||||
it under the terms of the GNU General Public License as published by |
||||
the Free Software Foundation, either version 3 of the License, or |
||||
(at your option) any later version. |
||||
|
||||
This program is distributed in the hope that it will be useful, |
||||
but WITHOUT ANY WARRANTY; without even the implied warranty of |
||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
||||
GNU General Public License for more details. |
||||
|
||||
You should have received a copy of the GNU General Public License |
||||
along with this program. If not, see <https://www.gnu.org/licenses/>. |
||||
|
||||
Also add information on how to contact you by electronic and paper mail. |
||||
|
||||
If the program does terminal interaction, make it output a short |
||||
notice like this when it starts in an interactive mode: |
||||
|
||||
<program> Copyright (C) <year> <name of author> |
||||
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. |
||||
This is free software, and you are welcome to redistribute it |
||||
under certain conditions; type `show c' for details. |
||||
|
||||
The hypothetical commands `show w' and `show c' should show the appropriate |
||||
parts of the General Public License. Of course, your program's commands |
||||
might be different; for a GUI interface, you would use an "about box". |
||||
|
||||
You should also get your employer (if you work as a programmer) or school, |
||||
if any, to sign a "copyright disclaimer" for the program, if necessary. |
||||
For more information on this, and how to apply and follow the GNU GPL, see |
||||
<https://www.gnu.org/licenses/>. |
||||
|
||||
The GNU General Public License does not permit incorporating your program |
||||
into proprietary programs. If your program is a subroutine library, you |
||||
may consider it more useful to permit linking proprietary applications with |
||||
the library. If this is what you want to do, use the GNU Lesser General |
||||
Public License instead of this License. But first, please read |
||||
<https://www.gnu.org/licenses/why-not-lgpl.html>. |
||||
@ -0,0 +1,700 @@ |
||||
[简体中文](README-CN.md) | [English](README.md) |
||||
|
||||
<div align="center"> |
||||
|
||||
# TypePHP |
||||
|
||||
**PHP 原生 AOT 编译器** |
||||
|
||||
将 PHP 源码提前(AOT)编译为原生机器码,生成原生可执行文件、PHP 扩展和共享库, |
||||
同时保留你熟悉的 PHP 语法。 |
||||
|
||||
[](https://github.com/swoole/typephp/actions/workflows/linux-x64.yml) |
||||
[](https://github.com/swoole/typephp/actions/workflows/linux-arm64.yml) |
||||
[](https://github.com/swoole/typephp/actions/workflows/macos-arm64.yml) |
||||
[](https://github.com/swoole/typephp/actions/workflows/windows-build.yml) |
||||
[](https://www.php.net/) |
||||
[](LICENSE) |
||||
|
||||
</div> |
||||
|
||||
--- |
||||
|
||||
## 什么是 TypePHP? |
||||
|
||||
TypePHP 是一个 AOT(Ahead-Of-Time,提前编译)编译器,它把 PHP 源码翻译为 C++, |
||||
再编译为原生机器码。与字节码缓存或虚拟机不同,它不会在运行时解释 opcode, |
||||
而是直接生成在 CPU 上运行的原生二进制。 |
||||
|
||||
它保留熟悉的 PHP 语法,同时引入编译期类型信息,让编译器为性能热点生成快速、 |
||||
静态类型的 C++ 代码。动态 PHP 值、内置函数、反射和对象元数据继续通过 PHPX |
||||
与 Zend runtime 互操作;用户函数编译完成后不再以 Zend opcode 方式执行。 |
||||
|
||||
TypePHP **完全由 PHP 语言编写**,并且**完全自举**:`tpc` 编译器二进制就是 |
||||
用 TypePHP 编译编译器自身的 PHP 源码得到的。整个自举链路是纯 PHP——编译器 |
||||
本身没有任何 C 或 C++ 胶水代码。 |
||||
|
||||
TypePHP 仍在积极开发中。它提供的是边界明确、可测试的 PHP 子集,而不是宣称可以 |
||||
无修改替代所有高度动态的 PHP 程序。在将现有项目迁移到 TypePHP 前,请先阅读 |
||||
[兼容性模型](#兼容性模型)和[不兼容特性清单](docs/zh-cn/INCOMPATIBLE_PHP_FEATURES.md)。 |
||||
|
||||
## 工作原理 |
||||
|
||||
```text |
||||
PHP 源码 + .stub.php 声明 + 可选 C/C++ 源码 |
||||
│ |
||||
▼ |
||||
解析、校验并收集全部声明 |
||||
│ |
||||
▼ |
||||
将函数实现和常量表达式降级为 C++17 |
||||
│ |
||||
▼ |
||||
原生编译器 + 可复用对象/PCH 缓存 |
||||
│ |
||||
▼ |
||||
可执行文件 | PHP 扩展 | 共享库 | WASI Component |
||||
``` |
||||
|
||||
prepare 阶段只建立完整符号模型,不分配运行时 Cache ID。常量和声明默认值只保留 |
||||
AST,待全部项目符号就绪后再在 convert 阶段解析。这一两阶段设计保证多文件构建和 |
||||
编译器自举过程具有确定性。 |
||||
|
||||
## 特性 |
||||
|
||||
- **完全自举、纯 PHP 实现** —— TypePHP 编译器完全由 PHP 语言编写,并能自举: |
||||
用 `tpc` 编译编译器自身的源码,即可生成原生二进制。 |
||||
- **真正的 AOT 编译** —— PHP 先降级为 C++17,再编译为原生机器码。无解释器、 |
||||
无 opcode 缓存、无 JIT 预热。 |
||||
- **三种原生构建模式** —— 同一份代码可编译为原生 `bin` 可执行文件、可加载的 |
||||
PHP `ext` 扩展,或可复用的 `lib` 共享库。 |
||||
- **原生类型系统** —— `int`、`float`、`bool` 直接映射为 C++ 标量类型 |
||||
(`int64_t`、`double`、`bool`),数值代码可获得数量级的性能提升。 |
||||
- **高精度数值** —— `bigInt`(GMP)、`decimal`(libmpdec)、`bigFloat`(MPFR), |
||||
提供强类型运算符和方法 API。 |
||||
- **强类型容器** —— `std::array`、`std::vector`、`std::map`、`std::ordered_map`, |
||||
元素类型在编译期确定;最高比 PHP 数组快 **10 倍**,性能与 C++ `std::vector` 相当。 |
||||
- **通用方法(Universal Methods)** —— 在原生类型上直接调用方法 |
||||
(`$s->upper()`、`$arr->contains()`、`$big->mul(2)`);静态类型已知时在编译期 |
||||
直接解析调用。 |
||||
- **混合 C++ / PHP 编程** —— 在性能关键内核中直接调用 C++ 函数(反之亦然)。 |
||||
- **编译期函数与关键词** —— `any()`、`refval()`、`objval()`、`expected()`、 |
||||
`unexpected()`,以及 `toInt()`、`toString()`、`toArray()` 等。 |
||||
- **编译期安全检查** —— `#[Immutable]` 只读契约和 `#[ArrayDef]` 数组结构元数据, |
||||
在编译期检查,零运行时开销。 |
||||
- **编译期代码生成** —— `#[Getter]`、`#[Setter]`、`#[With]`、`#[Constructor]`、 |
||||
`#[Printer]` 和 `#[Arrayable]` 根据属性声明生成类型安全的方法。 |
||||
- **现代 PHP 支持** —— PHP 8.4 property hooks、非对称可见性、PHP 8.5 |
||||
`clone()`-with 以及 `(void)` 丢弃表达式。 |
||||
- **跨平台与 WASM** —— 面向 x64 和 ARM64 的 Linux、Windows、macOS 目标, |
||||
以及 WASI 0.2 和浏览器(Jco)输出。 |
||||
- **Python 桥接** —— 为 Python 模块生成 IDE helper,并将 Python 脚本转换为 TypePHP。 |
||||
|
||||
## 为什么选择 TypePHP? |
||||
|
||||
| | TypePHP AOT | 字节码缓存(OPcache) | JIT(PHP 8+) | |
||||
|---|---|---|---| |
||||
| 编译目标 | 原生机器码 | 字节码 | 机器码(trace) | |
||||
| 启动 / 预热 | 无(已编译完成) | 每进程预热 | JIT 预热 | |
||||
| 类型驱动优化 | 编译期、全程序 | 无 | 有限,基于 trace | |
||||
| 生成原生可执行文件 | 支持 | 不支持 | 不支持 | |
||||
| 源码保护 | 编译为机器码 | 字节码(可还原) | 字节码(可还原) | |
||||
| 性能确定性 | 是 | 否 | 否 | |
||||
|
||||
**相较原生 PHP 的优势:** |
||||
|
||||
- **接近原生的性能。** 数值密集和容器密集的热点路径会编译为与 C++ 程序相同的机器码。 |
||||
见下方[基准测试](#基准测试)。 |
||||
- **源码保护。** 源码被编译掉——交付物是原生二进制,而不是可读的 PHP 文件。 |
||||
- **原生进程入口。** 二进制模式直接启动原生可执行文件,不需要 PHP CLI 或独立的 |
||||
解释器进程。可执行文件仍会嵌入或链接 PHPX、`libphp` 及项目配置的原生库,部署包 |
||||
中必须提供这些运行时依赖。 |
||||
- **渐进式类型,真正带来收益。** 只在性能关键处添加 `use native_types`、`std::` |
||||
容器和类型声明,其余保持普通 PHP。 |
||||
- **Zend 生态互通。** 扩展模式以标准 PHP 扩展形式加载,项目可以调用受支持的 |
||||
内置函数,并显式声明依赖的其他 Zend 扩展。 |
||||
|
||||
## 前置要求 |
||||
|
||||
- **PHP 8.4 – 8.5** CLI、开发头文件及 `php-config` |
||||
- 在类 Unix 系统构建二进制/共享库时,需要与 PHP 匹配的 **embed 库** |
||||
(`libphp.so` 或 `libphp.dylib`) |
||||
- **GCC 9+**(或 Clang),支持 **C++17** |
||||
- **CMake 3.24+** |
||||
- **Composer 2** |
||||
- 高精度数学库:**GMP**、**MPFR**(libmpdec 已随 PHPX 内置) |
||||
|
||||
```shell |
||||
# Ubuntu/Debian |
||||
sudo apt install build-essential cmake pkg-config libgmp-dev libmpfr-dev |
||||
|
||||
# RHEL/CentOS/Fedora |
||||
sudo dnf install gcc gcc-c++ cmake pkgconf-pkg-config gmp-devel mpfr-devel |
||||
|
||||
# Arch Linux |
||||
sudo pacman -S base-devel cmake pkgconf gmp mpfr |
||||
``` |
||||
|
||||
> GMP 用于 `bigInt`,MPFR 用于 `bigFloat`。`decimal` 底层是 libmpdec, |
||||
> 已随 PHPX 内置,无需单独安装。 |
||||
|
||||
Linux x64 是主要开发及全量测试 CI 平台。编译器也提供 Windows、macOS、ARM64 和 |
||||
WASI 后端;具体主机能否构建某个目标,仍取决于 PHP embed、工具链和第三方库是否 |
||||
可用。 |
||||
|
||||
原生 Release Assets 默认使用 PHP 8.5 ZTS 的最新版本构建,提供 Linux x64、Linux |
||||
ARM64、macOS ARM64 和 Windows x64 四个平台包;不提供原生 NTS 或 32 位 x86 包。 |
||||
Linux 与 macOS 包包含编译器和 production Composer 依赖,Windows 包则包含完整且 |
||||
匹配的 PHP/PHPX 运行时与 SDK。 |
||||
|
||||
## 安装 |
||||
|
||||
### 通过 Composer |
||||
|
||||
```bash |
||||
composer require --dev swoole/typephp |
||||
``` |
||||
|
||||
然后编译你的项目: |
||||
|
||||
```bash |
||||
vendor/bin/tpc.php project.yml |
||||
``` |
||||
|
||||
在 TypePHP 源码仓库中开发时,改用本地入口: |
||||
|
||||
```bash |
||||
bin/tpc.php project.yml |
||||
``` |
||||
|
||||
### 从源码安装 |
||||
|
||||
```bash |
||||
git clone https://github.com/swoole/typephp.git |
||||
cd typephp |
||||
composer install |
||||
php bin/tpc.php --help |
||||
``` |
||||
|
||||
可以使用 `PHPX_HOME` 指向独立的 PHPX 源码或安装目录。`PHP_HOME` 可以指向 PHP |
||||
embed 安装前缀;在类 Unix 系统中,该目录应包含 `bin/php-config`、PHP 头文件和 |
||||
`lib/libphp.so`。 |
||||
|
||||
### 构建 `libphp.so` |
||||
|
||||
二进制和共享库构建需要 PHP 的 `embed` SAPI。如果 Linux 上缺少 `libphp.so`, |
||||
`tpc.php` 可以交互式下载 PHP 源码并自动构建。PHP 扩展构建从宿主 SAPI 解析 Zend |
||||
符号,不能再加载第二份 `libphp`。详见[自动构建 libphp.so](docs/zh-cn/LIBPHP_INSTALLER.md)。 |
||||
|
||||
## 快速开始 |
||||
|
||||
创建 `hello.php`: |
||||
|
||||
```php |
||||
<?php |
||||
|
||||
function main(): void |
||||
{ |
||||
echo "Hello World!\n"; |
||||
var_dump(PHP_VERSION); |
||||
var_dump(php_uname()); |
||||
} |
||||
``` |
||||
|
||||
编译并运行: |
||||
|
||||
```bash |
||||
bin/tpc.php hello.php |
||||
./hello |
||||
``` |
||||
|
||||
输出示例(具体 PHP 版本和平台字符串取决于实际链接的运行时): |
||||
|
||||
``` |
||||
Hello World! |
||||
string(5) "8.x.x" |
||||
string(16) "Linux ..." |
||||
``` |
||||
|
||||
> 二进制模式需要全局 `main()` 函数。它可以声明为无参数,或 |
||||
> `main(int $argc, array $argv)` 以接收命令行参数,且必须返回 `void`。全局作用域 |
||||
> 不允许可执行语句;可执行代码必须位于函数或方法内。 |
||||
|
||||
## 编译模式 |
||||
|
||||
TypePHP 支持三种构建模式,通过 `-m` / `--mode` 选择: |
||||
|
||||
| 模式 | 参数 | 输出 | 需要 `main()` | 典型用途 | |
||||
|---|---|---|---|---| |
||||
| 二进制 | `-m bin`(默认) | 可执行文件 | 是 | CLI 工具、常驻服务、独立应用 | |
||||
| 扩展 | `-m ext` | PHP `.so` / `.dll` | 否 | 将编译后的函数和类加载到 PHP SAPI | |
||||
| 库 | `-m lib` | 共享库及自动生成的 `.stub.php` | 否 | 在其他项目中复用编译后的 TypePHP API | |
||||
|
||||
```bash |
||||
# 二进制(默认) |
||||
bin/tpc.php app.php -o myapp |
||||
|
||||
# PHP 扩展 |
||||
bin/tpc.php extension/ -m ext -o my_extension |
||||
|
||||
# 共享库,同时生成 mylib.stub.php |
||||
bin/tpc.php lib/ -m lib -o mylib |
||||
``` |
||||
|
||||
详见[编译模式](docs/zh-cn/COMPILATION_MODES.md)。 |
||||
|
||||
## 项目配置 |
||||
|
||||
多文件项目建议使用 `project.yml` 固化可复用的构建配置: |
||||
|
||||
```yaml |
||||
name: myapp |
||||
mode: bin |
||||
php-version: "8.5" |
||||
optimize: 2 |
||||
job: 8 |
||||
build-dir: build |
||||
cxx-std: c++17 |
||||
|
||||
sources: |
||||
- src |
||||
- cpp-src |
||||
- path: src/php85 |
||||
if: PHP_VERSION_ID >= 80500 |
||||
- path: src/windows |
||||
if: PHP_OS_FAMILY == "Windows" |
||||
|
||||
ignore: |
||||
- src/experimental |
||||
|
||||
include-paths: |
||||
- native/include |
||||
defines: |
||||
- FEATURE_FAST_PATH=1 |
||||
link-paths: |
||||
- native/lib |
||||
link-libs: |
||||
- curl |
||||
|
||||
# Zend 扩展依赖,不是原生链接库。 |
||||
# `extension-dependencies` 是等价长名称,两者不能同时使用。 |
||||
ext-deps: |
||||
- pdo_mysql |
||||
- curl |
||||
``` |
||||
|
||||
路径以 YAML 文件所在目录为基准。source 可以是文件或目录;条件 source 支持 |
||||
`PHP_VERSION`、`PHP_VERSION_ID` 和 `PHP_OS_FAMILY`。命令行参数优先于 YAML |
||||
中的同名配置。原生链接依赖应写入 `link-libs`;`ext-deps` 会生成 |
||||
`ZEND_MOD_REQUIRED`,缺少所需 PHP 扩展时由 Zend 拒绝加载模块。 |
||||
|
||||
构建目录保存生成的 C++、依赖对象和预编译头缓存。复用同一个构建目录可以显著加快 |
||||
增量构建;仅在确实需要重编 PHPX 公共对象时使用 `--force`。 |
||||
|
||||
全部项目配置项及命令行优先级详见[编译器命令行](docs/zh-cn/COMPILER_CLI.md)。 |
||||
|
||||
## 兼容性模型 |
||||
|
||||
TypePHP 会在适合 AOT 编译的范围内保持 PHP 语法和运行行为,同时有一些明确限制: |
||||
|
||||
- 全局作用域只允许声明,可执行语句必须位于函数或方法内; |
||||
- 二进制模式对 `main()` 使用严格签名; |
||||
- `use native_types` 会让标量声明使用固定原生存储,之后不能改为不兼容类型; |
||||
- 静态可确定的调用和属性会直接编译,受支持的动态操作则通过 PHPX/Zend runtime |
||||
fallback 执行; |
||||
- `.stub.php` 用于声明 C++ 或外部库 API,函数体必须为空,stub 文件禁止声明 |
||||
`#[Native]` 类; |
||||
- 部分高度动态的引用、声明、闭包和反射模式仍明确不支持。 |
||||
|
||||
兼容性边界属于公共契约,同时有正向和负向测试保护。请以 |
||||
[不兼容 PHP 特性清单](docs/zh-cn/INCOMPATIBLE_PHP_FEATURES.md)为当前准确列表,不要把 |
||||
README 未提及的行为默认理解为已支持。 |
||||
|
||||
## 编译期 Attribute 与代码生成 |
||||
|
||||
TypePHP 在 class lowering 阶段消费内置的代码生成 Attribute。生成的方法保留属性 |
||||
声明的类型,并与显式声明的方法一样参与名称冲突、继承关系和 final 方法检查。 |
||||
|
||||
| Attribute | 目标 | 生成的 API | |
||||
|---|---|---| |
||||
| `#[Getter]` | 实例属性,包括构造器提升属性 | `public function getName(): T` | |
||||
| `#[Setter]` | 可变实例属性,包括构造器提升属性 | `public function setName(T $name): void` | |
||||
| `#[With]` | 可变实例属性,包括构造器提升属性 | `public function withName(T $name): static`;克隆对象、修改副本并返回副本 | |
||||
| `#[Constructor]` | 普通实例属性声明 | 将属性加入自动生成的 public `__construct()` | |
||||
| `#[Printer]` | 具名类 | `public function __toString(): string` | |
||||
| `#[Arrayable]` | 具名类 | `public function toArray(): array` | |
||||
|
||||
```php |
||||
<?php |
||||
|
||||
#[Printer(fields: ['id', 'name'])] |
||||
#[Arrayable(fields: ['id', 'name'])] |
||||
final class User |
||||
{ |
||||
#[Constructor, Getter, With] |
||||
public int $id; |
||||
|
||||
#[Constructor, Getter, Setter] |
||||
public string $name = 'guest'; |
||||
} |
||||
|
||||
function main(): void |
||||
{ |
||||
$user = new User(7); |
||||
$user->setName('Alice'); |
||||
|
||||
$copy = $user->withId(8); |
||||
echo $user->getId(); // 7 |
||||
echo $copy->getId(); // 8 |
||||
echo $user; // User(id=7, name=Alice) |
||||
echo $user->toArray()['name']; |
||||
} |
||||
``` |
||||
|
||||
未指定 `fields` 时,`#[Printer]` 和 `#[Arrayable]` 使用当前类自身的 public 实例 |
||||
属性。位置参数写法 `#[Arrayable(['id'])]` 等价于 |
||||
`#[Arrayable(fields: ['id'])]`。 |
||||
|
||||
`#[Getter]`、`#[Setter]` 和 `#[With]` 不能用于 static 属性或带 property hook 的 |
||||
属性;`#[Setter]` 和 `#[With]` 还会拒绝 readonly 属性。类中已经显式声明 |
||||
`__construct()` 时不能使用 `#[Constructor]`,必填的构造属性必须位于带默认值的 |
||||
属性之前。生成的方法名若与已有方法冲突,或覆盖继承而来的 final 方法,编译期会 |
||||
直接报错。 |
||||
|
||||
## 使用示例 |
||||
|
||||
### 1. 原生类型 —— 编译期数值加速 |
||||
|
||||
```php |
||||
<?php |
||||
use native_types; |
||||
|
||||
function fib(int $n): int |
||||
{ |
||||
if ($n == 1 || $n == 2) { |
||||
return 1; |
||||
} |
||||
return fib($n - 1) + fib($n - 2); |
||||
} |
||||
|
||||
function main(int $argc, array $argv): void |
||||
{ |
||||
$n = (int)$argv[1]; |
||||
$begin = microtime(true); |
||||
echo fib($n) . "\n"; |
||||
echo "Time: " . (microtime(true) - $begin) . "\n"; |
||||
} |
||||
``` |
||||
|
||||
```bash |
||||
bin/tpc.php fib.php -O3 -o fib |
||||
./fib 30 |
||||
``` |
||||
|
||||
使用 `use native_types` 后,`int` 变量变为 C++ `int64_t`,算术运算直接编译为 |
||||
CPU 指令,而不是 ZendVM 调用。 |
||||
|
||||
### 2. 高精度数值 |
||||
|
||||
```php |
||||
<?php |
||||
declare(strict_types=1); |
||||
use native_types; |
||||
|
||||
function main(): void |
||||
{ |
||||
// 54 位整数 —— 自动识别并存储为 bigInt |
||||
$a = std::bigInt("123456789012345678901234567890123456789012345678901234"); |
||||
$b = std::bigInt("987654321098765432109876543210987654321098765432109876"); |
||||
|
||||
echo $a->add($b)->toString() . "\n"; // 精确计算,不会溢出 |
||||
|
||||
// 精确的十进制运算 —— 无二进制浮点误差 |
||||
$c = std::decimal("0.1")->add(std::decimal("0.2")); |
||||
echo $c->toString() . "\n"; // "0.3" |
||||
|
||||
// 256 位浮点数 |
||||
$pi = std::bigFloat("3.14159265358979323846264338327950288419716939937510"); |
||||
echo $pi->mul(2)->toString() . "\n"; |
||||
} |
||||
``` |
||||
|
||||
详见[高精度类型](docs/zh-cn/HIGH_PRECISION_TYPES.md)和[原生类型](docs/zh-cn/NATIVE_TYPES.md)。 |
||||
|
||||
### 3. 强类型容器 |
||||
|
||||
```php |
||||
<?php |
||||
use native_types; |
||||
|
||||
function main(): void |
||||
{ |
||||
$vector = std::vector(Type::Int); |
||||
|
||||
$vector[] = 1; |
||||
$vector[] = 2; |
||||
$vector[] = 3; |
||||
|
||||
$sum = 0; |
||||
foreach ($vector as $value) { |
||||
$sum += $value; |
||||
} |
||||
|
||||
echo $sum . "\n"; // 6 |
||||
echo $vector[1] . "\n"; // 2 |
||||
|
||||
// 固定 key/value 类型的映射 |
||||
$map = std::ordered_map(Type::String, Type::Int); |
||||
$map["a"] = 1; |
||||
$map["b"] = 2; |
||||
} |
||||
``` |
||||
|
||||
详见 [Std 容器](docs/zh-cn/STD_CONTAINERS.md)。 |
||||
|
||||
### 4. 通用方法 |
||||
|
||||
```php |
||||
<?php |
||||
|
||||
function main(): void |
||||
{ |
||||
$s = "hello world"; |
||||
echo $s->length() . "\n"; // strlen() |
||||
echo $s->upper() . "\n"; // strtoupper() |
||||
echo $s->substr(0, 5) . "\n"; // substr() |
||||
|
||||
$arr = [1, 3, 5, 7, 9]; |
||||
echo $arr->count() . "\n"; // count() |
||||
var_dump($arr->contains(3)); // in_array() |
||||
|
||||
$big = std::bigInt("12345678901234567890"); |
||||
echo $big->mul(2)->toString() . "\n"; |
||||
} |
||||
``` |
||||
|
||||
原生类型上的方法调用在编译期被解析为直接的 C/C++ 函数调用——没有虚表查找、 |
||||
没有反射、没有运行时派发。详见[通用方法](docs/zh-cn/UNIVERSAL_METHODS.md)。 |
||||
|
||||
### 5. 混合 C++ / PHP |
||||
|
||||
用 C++ 编写性能关键内核,并在 PHP 中调用: |
||||
|
||||
```cpp |
||||
// math.cpp |
||||
#include <phpx.h> |
||||
|
||||
using namespace php; |
||||
|
||||
Int php_fast_sum(Int a, Int b) { |
||||
return a + b; |
||||
} |
||||
``` |
||||
|
||||
```php |
||||
<?php |
||||
// math.stub.php —— 声明 C++ 函数签名 |
||||
function fast_sum(int $a, int $b): int {} |
||||
``` |
||||
|
||||
```php |
||||
<?php |
||||
function main(): void |
||||
{ |
||||
echo fast_sum(3, 4) . "\n"; // 7 |
||||
} |
||||
``` |
||||
|
||||
需要将 `math.cpp`、`math.stub.php` 和调用它的 PHP 源码加入同一个项目配置。 |
||||
C++ 符号的 `php_` 前缀属于 TypePHP callable ABI;stub 函数只提供类型元数据, |
||||
不能包含实际实现。 |
||||
|
||||
详见[混合 C++/PHP](docs/zh-cn/MIXED_CPP_PHP.md)。 |
||||
|
||||
## 基准测试 |
||||
|
||||
### PHP 语言基准(来自 php-src) |
||||
|
||||
TypePHP 使用 `-O3` 运行 PHP 源码树自带的官方 `bench.php` 与 |
||||
`micro_bench.php` 语言性能测试: |
||||
|
||||
| 基准 | 解释执行 PHP | TypePHP AOT(`-O3`) | 加速比 | |
||||
|---|---|---|---| |
||||
| `bench.php`(总计) | 5.034 秒 | **0.603 秒** | 约 8× | |
||||
| `micro_bench.php`(总计) | 13.045 秒 | **2.021 秒** | 约 6.5× | |
||||
|
||||
两项基准覆盖 PHP 语言核心性能——函数调用、对象属性访问、数组/哈希访问、 |
||||
字符串处理、控制流等。测试代码见 [`benchmark/bench.php`](benchmark/bench.php) 和 |
||||
[`benchmark/micro_bench.php`](benchmark/micro_bench.php)。其他专项性能回归测试 |
||||
统一放置在 [`benchmark/`](benchmark/) 目录中。 |
||||
|
||||
这些数字是项目测量快照,不是性能保证。PHP 版本、编译器、CPU、优化参数和已启用 |
||||
扩展都会影响结果;在用于部署决策前,应在同一机器上使用相同 workload 自行对比。 |
||||
|
||||
### std::array 对比 PHP 数组 |
||||
|
||||
一个 10000×100000 的元素累加循环,对比 PHP 数组、TypePHP `std::array` |
||||
与原生 C++: |
||||
|
||||
| 实现 | 耗时 | |
||||
|---|---| |
||||
| PHP 数组(JIT) | 67.6 秒 | |
||||
| `std::array`(TypePHP AOT) | **6.4 秒** | |
||||
| C++ `std::vector` | 6.2 秒 | |
||||
|
||||
在该 workload 中,`std::array` 比 PHP 数组快约 **10 倍**,并接近手写 C++ 结果。 |
||||
完整基准测试见 [Std 容器](docs/zh-cn/STD_CONTAINERS.md)。 |
||||
|
||||
## 命令行 |
||||
|
||||
```bash |
||||
bin/tpc.php <file|dir|project.yml> [options] [-- program-args...] |
||||
``` |
||||
|
||||
常用示例: |
||||
|
||||
```bash |
||||
# 编译单个文件 |
||||
bin/tpc.php app.php |
||||
|
||||
# 优化并运行,`--` 后的参数传给生成的程序 |
||||
bin/tpc.php app.php -O3 -r -- --flag value |
||||
|
||||
# 编译 project.yml 定义的项目 |
||||
bin/tpc.php project.yml -O2 -j 8 |
||||
|
||||
# 生成 PHP 扩展 |
||||
bin/tpc.php extension/ -m ext -o my_extension |
||||
|
||||
# 只生成 C++(跳过编译与链接) |
||||
bin/tpc.php app.php --dry --build-dir /tmp/typephp-build |
||||
|
||||
# 编译为 WASI 0.2 |
||||
bin/tpc.php --wasm app.php |
||||
|
||||
# 编译为浏览器目标(需要 jco) |
||||
bin/tpc.php --wasm=browser app.php |
||||
``` |
||||
|
||||
主要选项: |
||||
|
||||
| 选项 | 说明 | |
||||
|---|---| |
||||
| `-O <0-3>` | 优化级别(默认 `0`) | |
||||
| `-d`, `--debug` | 调试构建,带符号和源码跟踪 | |
||||
| `-o`, `--output <file>` | 输出文件名 | |
||||
| `-m`, `--mode <bin\|lib\|ext>` | 构建模式(默认 `bin`) | |
||||
| `-r`, `--run` | 构建成功后运行 | |
||||
| `-j`, `--job <num>` | 并行编译任务数(默认 `4`) | |
||||
| `-f`, `--force` | 不使用缓存,重新编译可复用 PHPX 对象 | |
||||
| `--build-dir <dir>` | 生成 C++ 与中间产物的目录 | |
||||
| `--dry` | 只生成 C++,跳过编译与链接 | |
||||
| `--php-version <8.4\|8.5>` | 接受的 PHP 语法版本 | |
||||
| `--cxx-std <ver>` | C++ 标准(如 `c++17`、`c++20`) | |
||||
| `--march <arch>` | 目标指令集(如 `native`) | |
||||
| `--target-platform <triple>` | 交叉编译目标 triple | |
||||
| `--lto` | 启用链接时优化 | |
||||
| `--sanitize <type>` | 启用 sanitizer(如 `address`) | |
||||
| `--profile` | 启用 Linux gperftools 性能分析 | |
||||
| `--format` | 使用 clang-format 格式化生成的 C++ | |
||||
| `--no-literal-strings` | 禁用字面量字符串表优化 | |
||||
| `--no-progress`, `--no-color` | 适合 CI 的输出控制 | |
||||
| `-I`, `-D`, `-L`, `-l` | 可重复指定的原生 include、define、库路径和链接库参数 | |
||||
|
||||
运行 `bin/tpc.php --help` 查看权威的最新参数列表。详见 |
||||
[编译器命令行](docs/zh-cn/COMPILER_CLI.md),包括 Bash 补全: |
||||
|
||||
```bash |
||||
source <(./tpc --generate-completion=bash) |
||||
``` |
||||
|
||||
## 常见问题 |
||||
|
||||
- **缺少 `libphp.so` / `libphp.dylib`:** 安装或编译与当前 PHP 匹配的 embed SAPI,设置 |
||||
`PHP_HOME`,或使用 `bin/tpc.php` 在 Linux 上提供的交互式安装流程。 |
||||
- **找不到 PHPX:** 将 `PHPX_HOME` 指向包含 `include/` 和 |
||||
`lib/libphpx.so`(或对应平台文件)的 PHPX 安装目录,并在编译项目前先构建 PHPX。 |
||||
- **启动崩溃或出现 ABI 错误:** PHP 头文件、`php-config`、`libphp` 和扩展 ABI |
||||
必须使用一致的 PHP 版本及 ZTS/NTS 模式,不能混用不同 PHP 构建产生的产物。 |
||||
- **增量构建异常缓慢:** 固定使用同一个 `--build-dir`,以复用对象和 PCH 缓存。 |
||||
当外层测试工具已经并行运行多个测试时,不要再设置过大的 `tpc -j`,避免并发数 |
||||
相乘后造成 CPU 和内存争用。 |
||||
- **使用 `bin/tpc.php` 可以编译,但自举 `tpc` 失败:** 必须用自举编译器复现。 |
||||
自举执行可能暴露 PHP-hosted 编译器不会经过的动态调用或 ABI 路径。 |
||||
|
||||
## Python 桥接 |
||||
|
||||
TypePHP 内置一个 Python 工具子模块,复用 `tpc` 入口: |
||||
|
||||
```shell |
||||
# 为 Python 模块生成 IDE helper |
||||
./tpc --gen-python-helper math |
||||
./tpc --gen-python-helper numpy --output-dir .ide-helper |
||||
|
||||
# 将 Python 脚本转换为 TypePHP |
||||
./tpc --convert-python-to-php script.py > script.php |
||||
``` |
||||
|
||||
详见 [Python 工具子模块](docs/zh-cn/python/tools.md)。 |
||||
|
||||
## 开发与测试 |
||||
|
||||
安装开发依赖并运行编译器单元测试: |
||||
|
||||
```bash |
||||
composer install |
||||
PHPX_HOME=/path/to/phpx vendor/bin/phpunit |
||||
``` |
||||
|
||||
PHPT 是端到端测试。必须先构建自举编译器,并显式传给测试工具;将 Zend PHP |
||||
可执行文件作为 `--compiler` 并不能验证实际交付的编译器: |
||||
|
||||
```bash |
||||
PHPX_HOME=/path/to/phpx php bin/tpc.php project.yml --job 2 --no-progress |
||||
php run-tests.php -q -j8 --compiler ./tpc tests/compiler |
||||
``` |
||||
|
||||
静态分析与从测试源码生成的覆盖矩阵是两项独立检查: |
||||
|
||||
```bash |
||||
composer analyse |
||||
php bin/analyze-test-coverage.php |
||||
php bin/analyze-test-coverage.php \ |
||||
--format=markdown --output=build/test-coverage.md --strict |
||||
``` |
||||
|
||||
覆盖工具分别报告 PHP 版本 × 特性 × 正向编译 × 运行语义 × 负向诊断,并列出实际 |
||||
出现的 php-parser AST 节点。它不会给出分母不明确的单一百分比。详见 |
||||
[测试覆盖分析工具](docs/zh-cn/TEST_COVERAGE_ANALYZER.md)。 |
||||
|
||||
GitHub Actions 会在 PHP 8.4 和 8.5 上分别运行 PHPUnit 与自举 PHPT。修改编译器 |
||||
内部规则或代码生成时应增加聚焦的 PHPUnit;运行输出或诊断可观察时还应增加 PHPT。 |
||||
|
||||
## 文档 |
||||
|
||||
- [快速入门](docs/zh-cn/QUICKSTART.md) —— 最小编译流程 |
||||
- [编译模式](docs/zh-cn/COMPILATION_MODES.md) —— `bin`、`ext`、`lib` |
||||
- [编译器命令行](docs/zh-cn/COMPILER_CLI.md) —— CLI 参数与项目配置 |
||||
- [不兼容 PHP 特性清单](docs/zh-cn/INCOMPATIBLE_PHP_FEATURES.md) —— 当前限制 |
||||
- [原生类型](docs/zh-cn/NATIVE_TYPES.md) —— 原生标量类型 |
||||
- [高精度类型](docs/zh-cn/HIGH_PRECISION_TYPES.md) —— BigInt / Decimal / BigFloat |
||||
- [Std 容器](docs/zh-cn/STD_CONTAINERS.md) —— 强类型容器 |
||||
- [通用方法](docs/zh-cn/UNIVERSAL_METHODS.md) —— 编译期方法解析 |
||||
- [编译期函数](docs/zh-cn/COMPILE_TIME_FUNCTIONS.md) —— `any()`、`refval()`、`objval()` 等 |
||||
- [混合 C++/PHP](docs/zh-cn/MIXED_CPP_PHP.md) —— C++/PHP 互操作 |
||||
- [`#[Immutable]`](docs/zh-cn/IMMUTABLE.md) —— 编译期只读契约 |
||||
- [`#[ArrayDef]`](docs/zh-cn/ARRAY_DEF.md) —— 强类型数组属性契约 |
||||
- [Property hooks](docs/zh-cn/PROPERTY_HOOKS.md) —— PHP 8.4 hook 降级和运行时元数据 |
||||
- [对象存储模型](docs/zh-cn/OBJECT_STORAGE_AND_PASSING_MODELS.md) —— Zend object、Box 与 Native class 边界 |
||||
- [Generator](docs/zh-cn/YIELD_GENERATOR.md) —— 生成器降级与生命周期 |
||||
- [测试覆盖分析工具](docs/zh-cn/TEST_COVERAGE_ANALYZER.md) —— AST 与特性证据矩阵 |
||||
- [WASI 构建](docs/zh-cn/WASI_BUILD.md) —— WASI 目标 |
||||
|
||||
## 授权协议 |
||||
|
||||
TypePHP 采用 [GNU General Public License v3.0](LICENSE) 授权。 |
||||
|
||||
## 社区 |
||||
|
||||
- 代码仓库:<https://github.com/swoole/typephp> |
||||
- 版权所有 © 2026 上海识沃网络科技有限公司(Swoole) |
||||
@ -1,84 +1,751 @@ |
||||
# 依赖 |
||||
- 编译器需要 PHP 8.4 以上版本;生成的扩展仍可面向 PHP 8.2~8.5 |
||||
- 需要 GCC-9 以上版本,支持 C++17 标准 |
||||
- 需要 CMake-3.24 以上版本 |
||||
- 需要高精度数学库:`GMP`、`MPFR`、`libmpdec` |
||||
[English](README.md) | [简体中文](README-CN.md) |
||||
|
||||
# Composer 安装 |
||||
<div align="center"> |
||||
|
||||
在项目中安装 TypePHP: |
||||
# TypePHP |
||||
|
||||
**A native AOT compiler for PHP** |
||||
|
||||
Compile PHP source code into native machine code ahead of time — producing |
||||
native executables, PHP extensions, and shared libraries — while keeping |
||||
the PHP syntax you already know. |
||||
|
||||
[](https://github.com/swoole/typephp/actions/workflows/linux-x64.yml) |
||||
[](https://github.com/swoole/typephp/actions/workflows/linux-arm64.yml) |
||||
[](https://github.com/swoole/typephp/actions/workflows/macos-arm64.yml) |
||||
[](https://github.com/swoole/typephp/actions/workflows/windows-build.yml) |
||||
[](https://www.php.net/) |
||||
[](LICENSE) |
||||
|
||||
</div> |
||||
|
||||
--- |
||||
|
||||
## What is TypePHP? |
||||
|
||||
TypePHP is an Ahead-Of-Time (AOT) compiler that translates PHP source code into |
||||
C++ and then into native machine code. Unlike a bytecode cache or a VM, it does |
||||
not interpret opcodes at runtime: it generates optimized native binaries that |
||||
run directly on the CPU. |
||||
|
||||
It keeps familiar PHP syntax and adds compile-time type information, so the |
||||
compiler can emit fast, statically-typed C++ for hot paths. Dynamic PHP values, |
||||
internal functions, reflection, and object metadata continue to interoperate |
||||
with the Zend runtime through PHPX; user functions are not executed as Zend |
||||
opcodes after they have been compiled. |
||||
|
||||
TypePHP is **written entirely in PHP** and is **fully self-hosting**: the `tpc` |
||||
compiler binary is built by compiling the compiler's own PHP source code with |
||||
TypePHP. The bootstrap chain is pure PHP — no C or C++ glue in the compiler |
||||
itself. |
||||
|
||||
TypePHP is under active development. It intentionally supports a defined, |
||||
testable subset of PHP rather than claiming drop-in compatibility with every |
||||
dynamic PHP program. Read [Compatibility model](#compatibility-model) and the |
||||
[incompatible-feature list](docs/en/INCOMPATIBLE_PHP_FEATURES.md) before adopting |
||||
it for an existing application. |
||||
|
||||
## How it works |
||||
|
||||
```text |
||||
PHP source + .stub.php declarations + optional C/C++ sources |
||||
│ |
||||
▼ |
||||
parse, validate, and collect declarations |
||||
│ |
||||
▼ |
||||
lower function bodies and constants to C++17 |
||||
│ |
||||
▼ |
||||
native compiler + reusable object/PCH caches |
||||
│ |
||||
▼ |
||||
executable | PHP extension | shared library | WASI component |
||||
``` |
||||
|
||||
The prepare phase builds the complete symbol model without allocating runtime |
||||
cache IDs. Constants and declaration defaults retain their AST until the |
||||
convert phase, where they are lowered after all project symbols are known. |
||||
This two-phase design keeps multi-file and self-hosted builds deterministic. |
||||
|
||||
## Features |
||||
|
||||
- **Self-hosting, written in PHP** — the TypePHP compiler is implemented |
||||
entirely in PHP and bootstraps itself: `tpc` compiles the compiler's own |
||||
source into a native binary. |
||||
- **True AOT compilation** — PHP is lowered to C++17, then to native machine |
||||
code. No interpreter, no opcode cache, no JIT warm-up. |
||||
- **Three native build modes** — build a native `bin` executable, a loadable |
||||
PHP `ext` extension, or a reusable `lib` shared library from the same codebase. |
||||
- **Native type system** — `int`, `float`, and `bool` map directly to C++ |
||||
scalar types (`int64_t`, `double`, `bool`) for orders-of-magnitude speedups |
||||
on numeric code. |
||||
- **High-precision numerics** — `bigInt` (GMP), `decimal` (libmpdec), and |
||||
`bigFloat` (MPFR), with typed operators and method APIs. |
||||
- **Strongly-typed containers** — `std::array`, `std::vector`, `std::map`, and |
||||
`std::ordered_map` with compile-time element types; up to **10×** faster than |
||||
PHP arrays and on par with C++ `std::vector`. |
||||
- **Universal methods** — call methods directly on primitives |
||||
(`$s->upper()`, `$arr->contains()`, `$big->mul(2)`); statically-known calls |
||||
are resolved directly at compile time. |
||||
- **Mixed C++ / PHP** — call C++ functions from PHP (and vice versa) for |
||||
performance-critical kernels. |
||||
- **Compile-time functions & keywords** — `any()`, `refval()`, `objval()`, |
||||
`expected()`, `unexpected()`, plus `toInt()`, `toString()`, `toArray()` and |
||||
friends. |
||||
- **Compile-time safety** — `#[Immutable]` read-only contracts and `#[ArrayDef]` |
||||
array-shape metadata, checked at compile time with zero runtime cost. |
||||
- **Compile-time code generation** — `#[Getter]`, `#[Setter]`, `#[With]`, |
||||
`#[Constructor]`, `#[Printer]`, and `#[Arrayable]` generate type-safe methods |
||||
from property declarations. |
||||
- **Modern PHP support** — PHP 8.4 property hooks, asymmetric visibility, |
||||
PHP 8.5 `clone()`-with, and `(void)` discard expressions. |
||||
- **Cross-platform & WASM** — Linux, Windows, and macOS targets for x64 and |
||||
ARM64, plus WASI 0.2 and browser (Jco) output. |
||||
- **Python bridge** — generate IDE helpers for Python modules and convert |
||||
Python scripts to TypePHP. |
||||
|
||||
## Why TypePHP? |
||||
|
||||
| | TypePHP AOT | Opcode cache (OPcache) | JIT (PHP 8+) | |
||||
|---|---|---|---| |
||||
| Compilation target | Native machine code | Bytecode | Machine code (trace) | |
||||
| Startup / warm-up | None (already compiled) | Per-process warm-up | JIT warm-up | |
||||
| Type-driven optimization | Compile-time, full-program | None | Limited, trace-based | |
||||
| Native executable output | Yes | No | No | |
||||
| Source code protection | Compiled to machine code | Bytecode (reversible) | Bytecode (reversible) | |
||||
| Deterministic performance | Yes | No | No | |
||||
|
||||
**Strengths over plain PHP:** |
||||
|
||||
- **Near-native performance.** Numeric and container-heavy hot paths compile |
||||
down to the same machine code a C++ program would produce. See the |
||||
[benchmark](#benchmark) below. |
||||
- **Source protection.** Your source is compiled away — shipped artifacts are |
||||
native binaries, not readable PHP files. |
||||
- **Native process entry.** Binary mode starts directly from a native |
||||
executable and does not require the PHP CLI or a separate interpreter |
||||
process. The executable still embeds/links PHPX, `libphp`, and any configured |
||||
native libraries, which must be available in the deployment package. |
||||
- **Gradual typing that actually pays off.** Add `use native_types`, `std::` |
||||
containers, and type declarations only where performance matters; the rest |
||||
stays ordinary PHP. |
||||
- **Zend ecosystem interop.** Extension mode loads as a standard PHP extension, |
||||
and projects can call supported internal functions and require other Zend |
||||
extensions explicitly. |
||||
|
||||
## Requirements |
||||
|
||||
- **PHP 8.4 – 8.5** CLI, development headers, and `php-config` |
||||
- The matching **PHP embed library** (`libphp.so` or `libphp.dylib`) for binary/shared-library |
||||
builds on Unix-like systems |
||||
- **GCC 9+** (or Clang) with **C++17** |
||||
- **CMake 3.24+** |
||||
- **Composer 2** |
||||
- High-precision math libraries: **GMP**, **MPFR** (libmpdec is bundled with PHPX) |
||||
|
||||
```shell |
||||
# Ubuntu/Debian |
||||
sudo apt install build-essential cmake pkg-config libgmp-dev libmpfr-dev |
||||
|
||||
# RHEL/CentOS/Fedora |
||||
sudo dnf install gcc gcc-c++ cmake pkgconf-pkg-config gmp-devel mpfr-devel |
||||
|
||||
# Arch Linux |
||||
sudo pacman -S base-devel cmake pkgconf gmp mpfr |
||||
``` |
||||
|
||||
> GMP powers `bigInt` and MPFR powers `bigFloat`. The `decimal` type is backed |
||||
> by libmpdec, which is bundled with PHPX — no separate install required. |
||||
|
||||
Linux x64 is the primary development and full-test CI platform. The compiler |
||||
also has Windows, macOS, ARM64, and WASI backends; availability of PHP embed, |
||||
toolchain, and third-party libraries still determines which target can be |
||||
built on a given host. |
||||
|
||||
Native release assets are built with the latest PHP 8.5 ZTS release. TypePHP |
||||
publishes Linux x64, Linux ARM64, macOS ARM64, and Windows x64 packages. Native |
||||
NTS and 32-bit x86 packages are not provided. Linux and macOS archives contain |
||||
the compiler and production Composer dependencies, while the Windows archive |
||||
contains the complete matching PHP/PHPX runtime and SDK. |
||||
|
||||
## Installation |
||||
|
||||
### Via Composer |
||||
|
||||
```bash |
||||
composer require --dev swoole/typephp |
||||
``` |
||||
|
||||
安装后可直接编译项目: |
||||
Then compile your project: |
||||
|
||||
```bash |
||||
vendor/bin/tpc.php project.yml |
||||
``` |
||||
|
||||
在 TypePHP 源码仓库中则使用: |
||||
When working inside the TypePHP source repository, use the local entry point |
||||
instead: |
||||
|
||||
```bash |
||||
bin/tpc.php project.yml |
||||
``` |
||||
|
||||
Linux 环境缺少 `libphp.so` 时,`tpc.php` 可以交互式下载 PHP 源码并自动构建,详见 [自动构建 libphp.so](docs/LIBPHP_INSTALLER.md)。 |
||||
### From source |
||||
|
||||
```shell |
||||
# Ubuntu/Debian |
||||
sudo apt install libgmp-dev libmpfr-dev libmpdec-dev |
||||
```bash |
||||
git clone https://github.com/swoole/typephp.git |
||||
cd typephp |
||||
composer install |
||||
php bin/tpc.php --help |
||||
``` |
||||
|
||||
# RHEL/CentOS/Fedora |
||||
sudo dnf install gmp-devel mpfr-devel libmpdec-devel |
||||
`PHPX_HOME` may point to a separate PHPX checkout or installation. `PHP_HOME` |
||||
may point to the PHP embed prefix; it must contain `bin/php-config`, PHP headers, |
||||
and `lib/libphp.so` on Unix-like systems. |
||||
|
||||
# Arch Linux |
||||
sudo pacman -S gmp mpfr mpdecimal |
||||
### Building `libphp.so` |
||||
|
||||
Binary and shared-library builds require PHP's `embed` SAPI. If `libphp.so` is |
||||
missing on Linux, `tpc.php` can interactively download the PHP source and build |
||||
it for you. A PHP extension build resolves Zend symbols from the host SAPI and |
||||
must not load a second `libphp`. See |
||||
[Automatic libphp.so build](docs/en/LIBPHP_INSTALLER.md). |
||||
|
||||
## Quick Start |
||||
|
||||
Create `hello.php`: |
||||
|
||||
```php |
||||
<?php |
||||
|
||||
function main(): void |
||||
{ |
||||
echo "Hello World!\n"; |
||||
var_dump(PHP_VERSION); |
||||
var_dump(php_uname()); |
||||
} |
||||
``` |
||||
|
||||
> GMP 用于 `BigInt` 任意精度整数,MPFR 用于 `BigFloat` 高精度浮点数,libmpdec 用于 `Decimal` 十进制高精度小数。 |
||||
Compile and run it: |
||||
|
||||
> 预览版目前仅支持 `Linux` 系统,建议使用 `Ubuntu 22.04` |
||||
```bash |
||||
bin/tpc.php hello.php |
||||
./hello |
||||
``` |
||||
|
||||
## PHP |
||||
必须包含 embed 模块 |
||||
Example output (the exact PHP version and platform strings depend on the linked |
||||
runtime): |
||||
|
||||
## PHPX |
||||
可使用 `composer install` 安装依赖。 |
||||
进入 `vendor/swoole/phpx` 目录,编译 `phpx` |
||||
``` |
||||
Hello World! |
||||
string(5) "8.x.x" |
||||
string(16) "Linux ..." |
||||
``` |
||||
|
||||
```shell |
||||
cd vendor/swoole/phpx |
||||
cmake . |
||||
make -j32 |
||||
> Binary mode requires a global `main()` function. It may be declared with no |
||||
> parameters, or as `main(int $argc, array $argv)` to receive command-line |
||||
> arguments, and must return `void`. Top-level executable statements are not |
||||
> allowed; executable code belongs in a function or method. |
||||
|
||||
## Compilation Modes |
||||
|
||||
TypePHP supports three build modes, selected with `-m` / `--mode`: |
||||
|
||||
| Mode | Flag | Output | Needs `main()` | Typical use | |
||||
|---|---|---|---|---| |
||||
| Binary | `-m bin` (default) | Executable | Yes | CLI tools, long-running services, standalone apps | |
||||
| Extension | `-m ext` | PHP `.so` / `.dll` | No | Loading compiled functions/classes into a PHP SAPI | |
||||
| Library | `-m lib` | Shared library plus generated `.stub.php` | No | Reusing a compiled TypePHP API from another project | |
||||
|
||||
```bash |
||||
# Binary (default) |
||||
bin/tpc.php app.php -o myapp |
||||
|
||||
# PHP extension |
||||
bin/tpc.php extension/ -m ext -o my_extension |
||||
|
||||
# Shared library; also generates mylib.stub.php |
||||
bin/tpc.php lib/ -m lib -o mylib |
||||
``` |
||||
|
||||
## 动态链接库 |
||||
```shell |
||||
sudo ldconfig -p | grep php |
||||
See [Compilation modes](docs/en/COMPILATION_MODES.md) for details. |
||||
|
||||
## Project configuration |
||||
|
||||
For multi-file projects, keep repeatable build settings in `project.yml`: |
||||
|
||||
```yaml |
||||
name: myapp |
||||
mode: bin |
||||
php-version: "8.5" |
||||
optimize: 2 |
||||
job: 8 |
||||
build-dir: build |
||||
cxx-std: c++17 |
||||
|
||||
sources: |
||||
- src |
||||
- cpp-src |
||||
- path: src/php85 |
||||
if: PHP_VERSION_ID >= 80500 |
||||
- path: src/windows |
||||
if: PHP_OS_FAMILY == "Windows" |
||||
|
||||
ignore: |
||||
- src/experimental |
||||
|
||||
include-paths: |
||||
- native/include |
||||
defines: |
||||
- FEATURE_FAST_PATH=1 |
||||
link-paths: |
||||
- native/lib |
||||
link-libs: |
||||
- curl |
||||
|
||||
# Zend extension requirements, not native linker libraries. |
||||
# `extension-dependencies` is the equivalent long name; do not use both. |
||||
ext-deps: |
||||
- pdo_mysql |
||||
- curl |
||||
``` |
||||
必须包含 `libphp.so` 和 `libphpx.so` |
||||
|
||||
若编译完成,但找不到动态链接库,需要修改 |
||||
```shell |
||||
vim /etc/ld.so.conf.d/swoole.conf |
||||
Paths are resolved relative to the YAML file. A source entry may be a file or |
||||
directory; conditional entries support `PHP_VERSION`, `PHP_VERSION_ID`, and |
||||
`PHP_OS_FAMILY`. CLI arguments override their YAML counterparts. Native linker |
||||
dependencies belong in `link-libs`; `ext-deps` writes `ZEND_MOD_REQUIRED` |
||||
entries so Zend can reject loading when a required PHP extension is missing. |
||||
|
||||
The build directory contains generated C++, dependency objects, and the |
||||
precompiled-header cache. Reusing it makes incremental builds much faster; |
||||
use `--force` only when the reusable PHPX objects must be rebuilt. |
||||
|
||||
See [Compiler CLI](docs/en/COMPILER_CLI.md) for all project keys and command-line |
||||
precedence rules. |
||||
|
||||
## Compatibility model |
||||
|
||||
TypePHP follows PHP syntax and runtime behavior where they are compatible with |
||||
ahead-of-time compilation, but it also makes several deliberate restrictions: |
||||
|
||||
- global scope is declaration-only; executable statements must be inside a |
||||
function or method; |
||||
- binary mode has a strict `main()` signature; |
||||
- `use native_types` opts scalar declarations into fixed native storage, so a |
||||
value cannot later change to an incompatible type; |
||||
- statically-known calls and properties are compiled directly, while supported |
||||
dynamic operations use PHPX/Zend runtime fallbacks; |
||||
- `.stub.php` files declare C++ or imported-library APIs and must contain empty |
||||
bodies; `#[Native]` classes are not permitted in stub files; |
||||
- some highly dynamic reference, declaration, closure, and reflection patterns |
||||
remain intentionally unsupported. |
||||
|
||||
The compatibility boundary is part of the public contract and has both |
||||
positive and negative tests. Consult |
||||
[Incompatible PHP features](docs/en/INCOMPATIBLE_PHP_FEATURES.md) for the current, |
||||
specific list instead of assuming that absence from this README means support. |
||||
|
||||
## Compile-time attributes and code generation |
||||
|
||||
TypePHP consumes its built-in code-generation attributes while lowering the |
||||
class. The generated methods retain the declared property types and take part |
||||
in the same conflict, inheritance, and final-method checks as explicitly |
||||
declared methods. |
||||
|
||||
| Attribute | Target | Generated API | |
||||
|---|---|---| |
||||
| `#[Getter]` | Instance property, including a promoted property | `public function getName(): T` | |
||||
| `#[Setter]` | Mutable instance property, including a promoted property | `public function setName(T $name): void` | |
||||
| `#[With]` | Mutable instance property, including a promoted property | `public function withName(T $name): static`; clones the object, updates the clone, and returns it | |
||||
| `#[Constructor]` | Declared instance property | Adds the property to a generated public `__construct()` | |
||||
| `#[Printer]` | Named class | `public function __toString(): string` | |
||||
| `#[Arrayable]` | Named class | `public function toArray(): array` | |
||||
|
||||
```php |
||||
<?php |
||||
|
||||
#[Printer(fields: ['id', 'name'])] |
||||
#[Arrayable(fields: ['id', 'name'])] |
||||
final class User |
||||
{ |
||||
#[Constructor, Getter, With] |
||||
public int $id; |
||||
|
||||
#[Constructor, Getter, Setter] |
||||
public string $name = 'guest'; |
||||
} |
||||
|
||||
function main(): void |
||||
{ |
||||
$user = new User(7); |
||||
$user->setName('Alice'); |
||||
|
||||
$copy = $user->withId(8); |
||||
echo $user->getId(); // 7 |
||||
echo $copy->getId(); // 8 |
||||
echo $user; // User(id=7, name=Alice) |
||||
echo $user->toArray()['name']; |
||||
} |
||||
``` |
||||
|
||||
Without `fields`, `#[Printer]` and `#[Arrayable]` use the class's own public |
||||
instance properties. The positional form, such as `#[Arrayable(['id'])]`, is |
||||
equivalent to `#[Arrayable(fields: ['id'])]`. |
||||
|
||||
`#[Getter]`, `#[Setter]`, and `#[With]` cannot target static properties or |
||||
properties with hooks. `#[Setter]` and `#[With]` additionally reject readonly |
||||
properties. `#[Constructor]` cannot be used when the class already declares |
||||
`__construct()`, and required constructor properties must precede properties |
||||
with defaults. A generated method name that conflicts with a declared or |
||||
inherited final method is a compile-time error. |
||||
|
||||
## Examples |
||||
|
||||
### 1. Native types — compile-time numeric speedup |
||||
|
||||
```php |
||||
<?php |
||||
use native_types; |
||||
|
||||
function fib(int $n): int |
||||
{ |
||||
if ($n == 1 || $n == 2) { |
||||
return 1; |
||||
} |
||||
return fib($n - 1) + fib($n - 2); |
||||
} |
||||
|
||||
function main(int $argc, array $argv): void |
||||
{ |
||||
$n = (int)$argv[1]; |
||||
$begin = microtime(true); |
||||
echo fib($n) . "\n"; |
||||
echo "Time: " . (microtime(true) - $begin) . "\n"; |
||||
} |
||||
``` |
||||
|
||||
```bash |
||||
bin/tpc.php fib.php -O3 -o fib |
||||
./fib 30 |
||||
``` |
||||
|
||||
With `use native_types`, `int` variables become C++ `int64_t` and arithmetic |
||||
compiles to plain CPU instructions instead of ZendVM calls. |
||||
|
||||
### 2. High-precision numerics |
||||
|
||||
```php |
||||
<?php |
||||
declare(strict_types=1); |
||||
use native_types; |
||||
|
||||
function main(): void |
||||
{ |
||||
// 54-digit integer — automatically detected and stored as bigInt |
||||
$a = std::bigInt("123456789012345678901234567890123456789012345678901234"); |
||||
$b = std::bigInt("987654321098765432109876543210987654321098765432109876"); |
||||
|
||||
echo $a->add($b)->toString() . "\n"; // exact, no overflow |
||||
|
||||
// Exact decimal arithmetic — no binary floating-point error |
||||
$c = std::decimal("0.1")->add(std::decimal("0.2")); |
||||
echo $c->toString() . "\n"; // "0.3" |
||||
|
||||
// 256-bit floating point |
||||
$pi = std::bigFloat("3.14159265358979323846264338327950288419716939937510"); |
||||
echo $pi->mul(2)->toString() . "\n"; |
||||
} |
||||
``` |
||||
|
||||
See [High-precision types](docs/en/HIGH_PRECISION_TYPES.md) and |
||||
[Native types](docs/en/NATIVE_TYPES.md). |
||||
|
||||
### 3. Strongly-typed containers |
||||
|
||||
```php |
||||
<?php |
||||
use native_types; |
||||
|
||||
function main(): void |
||||
{ |
||||
$vector = std::vector(Type::Int); |
||||
|
||||
$vector[] = 1; |
||||
$vector[] = 2; |
||||
$vector[] = 3; |
||||
|
||||
$sum = 0; |
||||
foreach ($vector as $value) { |
||||
$sum += $value; |
||||
} |
||||
|
||||
echo $sum . "\n"; // 6 |
||||
echo $vector[1] . "\n"; // 2 |
||||
|
||||
// key-value map with fixed key/value types |
||||
$map = std::ordered_map(Type::String, Type::Int); |
||||
$map["a"] = 1; |
||||
$map["b"] = 2; |
||||
} |
||||
``` |
||||
|
||||
See [Std containers](docs/en/STD_CONTAINERS.md). |
||||
|
||||
### 4. Universal methods |
||||
|
||||
```php |
||||
<?php |
||||
|
||||
function main(): void |
||||
{ |
||||
$s = "hello world"; |
||||
echo $s->length() . "\n"; // strlen() |
||||
echo $s->upper() . "\n"; // strtoupper() |
||||
echo $s->substr(0, 5) . "\n"; // substr() |
||||
|
||||
$arr = [1, 3, 5, 7, 9]; |
||||
echo $arr->count() . "\n"; // count() |
||||
var_dump($arr->contains(3)); // in_array() |
||||
|
||||
$big = std::bigInt("12345678901234567890"); |
||||
echo $big->mul(2)->toString() . "\n"; |
||||
} |
||||
``` |
||||
|
||||
Method calls on primitives are resolved at compile time into direct C/C++ |
||||
function calls — no vtable lookup, no reflection, no runtime dispatch. See |
||||
[Universal methods](docs/en/UNIVERSAL_METHODS.md). |
||||
|
||||
### 5. Mixed C++ / PHP |
||||
|
||||
Write performance-critical kernels in C++ and call them from PHP: |
||||
|
||||
```cpp |
||||
// math.cpp |
||||
#include <phpx.h> |
||||
|
||||
using namespace php; |
||||
|
||||
Int php_fast_sum(Int a, Int b) { |
||||
return a + b; |
||||
} |
||||
``` |
||||
|
||||
```php |
||||
<?php |
||||
// math.stub.php — declares the C++ function signature |
||||
function fast_sum(int $a, int $b): int {} |
||||
``` |
||||
|
||||
```php |
||||
<?php |
||||
function main(): void |
||||
{ |
||||
echo fast_sum(3, 4) . "\n"; // 7 |
||||
} |
||||
``` |
||||
|
||||
Add `math.cpp`, `math.stub.php`, and the calling PHP source to the same project |
||||
configuration. The `php_` C++ symbol prefix is the TypePHP callable ABI; stub |
||||
functions provide type metadata only and must not contain an implementation. |
||||
|
||||
See [Mixed C++/PHP](docs/en/MIXED_CPP_PHP.md). |
||||
|
||||
## Benchmark |
||||
|
||||
### PHP language benchmarks (from php-src) |
||||
|
||||
TypePHP runs the official `bench.php` and `micro_bench.php` language |
||||
benchmarks that ship with the PHP source tree, compiled with `-O3`: |
||||
|
||||
| Benchmark | Interpreted PHP | TypePHP AOT (`-O3`) | Speedup | |
||||
|---|---|---|---| |
||||
| `bench.php` (total) | 5.034 s | **0.603 s** | ~8× | |
||||
| `micro_bench.php` (total) | 13.045 s | **2.021 s** | ~6.5× | |
||||
|
||||
Both benchmarks measure core PHP language performance — function calls, object |
||||
property access, array/hash access, string handling, control flow, and more. |
||||
The checked-in workloads are [`benchmark/bench.php`](benchmark/bench.php) and |
||||
[`benchmark/micro_bench.php`](benchmark/micro_bench.php). Additional focused |
||||
performance regressions live in the same [`benchmark/`](benchmark/) directory. |
||||
|
||||
These numbers are a project measurement snapshot, not a performance guarantee. |
||||
PHP version, compiler, CPU, optimization flags, and enabled extensions can all |
||||
change the result; compare on the same machine with the same workload before |
||||
making deployment decisions. |
||||
|
||||
### std::array vs PHP array |
||||
|
||||
A 10000×100000 element update loop, comparing PHP arrays against TypePHP's |
||||
`std::array` and native C++: |
||||
|
||||
| Implementation | Time | |
||||
|---|---| |
||||
| PHP array (JIT) | 67.6 s | |
||||
| `std::array` (TypePHP AOT) | **6.4 s** | |
||||
| C++ `std::vector` | 6.2 s | |
||||
|
||||
`std::array` is roughly **10× faster** than PHP arrays and performs |
||||
close to the hand-written C++ result in this workload. See the benchmark in |
||||
[Std containers](docs/en/STD_CONTAINERS.md). |
||||
|
||||
## Command Line |
||||
|
||||
```bash |
||||
bin/tpc.php <file|dir|project.yml> [options] [-- program-args...] |
||||
``` |
||||
|
||||
添加路径 |
||||
Common usage: |
||||
|
||||
```bash |
||||
# Compile a single file |
||||
bin/tpc.php app.php |
||||
|
||||
# Optimize and run, passing args to the program after `--` |
||||
bin/tpc.php app.php -O3 -r -- --flag value |
||||
|
||||
# Compile a project defined in project.yml |
||||
bin/tpc.php project.yml -O2 -j 8 |
||||
|
||||
# Build a PHP extension |
||||
bin/tpc.php extension/ -m ext -o my_extension |
||||
|
||||
# Only generate C++ (skip compile & link) |
||||
bin/tpc.php app.php --dry --build-dir /tmp/typephp-build |
||||
|
||||
# Compile to WASI 0.2 |
||||
bin/tpc.php --wasm app.php |
||||
|
||||
# Compile for the browser (requires jco) |
||||
bin/tpc.php --wasm=browser app.php |
||||
``` |
||||
/home/swoole/workspace/projects/phpx/lib |
||||
/opt/php-8.4/lib/ |
||||
|
||||
Key options: |
||||
|
||||
| Option | Description | |
||||
|---|---| |
||||
| `-O <0-3>` | Optimization level (default `0`) | |
||||
| `-d`, `--debug` | Debug build with symbols and source tracking | |
||||
| `-o`, `--output <file>` | Output file name | |
||||
| `-m`, `--mode <bin\|lib\|ext>` | Build mode (default `bin`) | |
||||
| `-r`, `--run` | Run after a successful build | |
||||
| `-j`, `--job <num>` | Parallel compile jobs (default `4`) | |
||||
| `-f`, `--force` | Rebuild reusable PHPX objects instead of using the cache | |
||||
| `--build-dir <dir>` | Directory for generated C++ and intermediates | |
||||
| `--dry` | Generate C++ only, skip compile and link | |
||||
| `--php-version <8.4\|8.5>` | PHP syntax version to accept | |
||||
| `--cxx-std <ver>` | C++ standard (e.g. `c++17`, `c++20`) | |
||||
| `--march <arch>` | Target instruction set (e.g. `native`) | |
||||
| `--target-platform <triple>` | Cross-compilation target triple | |
||||
| `--lto` | Enable link-time optimization | |
||||
| `--sanitize <type>` | Enable a sanitizer (e.g. `address`) | |
||||
| `--profile` | Enable Linux gperftools profiling | |
||||
| `--format` | Format generated C++ with clang-format | |
||||
| `--no-literal-strings` | Disable the literal-string table optimization | |
||||
| `--no-progress`, `--no-color` | CI-friendly output controls | |
||||
| `-I`, `-D`, `-L`, `-l` | Repeatable native include, define, library path, and library options | |
||||
|
||||
Run `bin/tpc.php --help` for the authoritative, up-to-date list. See |
||||
[Compiler CLI](docs/en/COMPILER_CLI.md) for details, including Bash completion: |
||||
|
||||
```bash |
||||
source <(./tpc --generate-completion=bash) |
||||
``` |
||||
|
||||
## Release packaging |
||||
## Troubleshooting |
||||
|
||||
- **`libphp.so` / `libphp.dylib` is missing:** install/build the matching PHP embed SAPI, set |
||||
`PHP_HOME`, or let `bin/tpc.php` offer the interactive Linux installer. |
||||
- **PHPX cannot be found:** set `PHPX_HOME` to a PHPX installation containing |
||||
`include/` and `lib/libphpx.so` (or the platform equivalent), then build PHPX |
||||
before compiling the project. |
||||
- **Startup crashes or ABI errors:** the PHP headers, `php-config`, `libphp`, |
||||
and loaded extension ABI must agree on the PHP version and ZTS/NTS mode. Do |
||||
not mix artifacts from different PHP builds. |
||||
- **Incremental builds are unexpectedly slow:** keep a stable `--build-dir` so |
||||
object and PCH caches can be reused. When an external test runner already |
||||
runs several tests concurrently, avoid multiplying that concurrency by an |
||||
unnecessarily large `tpc -j` value. |
||||
- **A project compiles with `bin/tpc.php` but fails with `tpc`:** reproduce with |
||||
the self-hosted compiler. Bootstrap execution can expose dynamic-call or ABI |
||||
paths that the PHP-hosted compiler does not exercise. |
||||
|
||||
## Python bridge |
||||
|
||||
Use the same PHP entry point on Windows, Linux, and macOS: |
||||
TypePHP ships a Python tool submodule that shares the `tpc` entry point: |
||||
|
||||
```shell |
||||
php package.php |
||||
# Generate IDE helpers for Python modules |
||||
./tpc --gen-python-helper math |
||||
./tpc --gen-python-helper numpy --output-dir .ide-helper |
||||
|
||||
# Convert a Python script to TypePHP |
||||
./tpc --convert-python-to-php script.py > script.php |
||||
``` |
||||
|
||||
See [Python tool submodule](docs/en/python/tools.md). |
||||
|
||||
## Development and testing |
||||
|
||||
Install development dependencies and run the compiler unit suite: |
||||
|
||||
```bash |
||||
composer install |
||||
PHPX_HOME=/path/to/phpx vendor/bin/phpunit |
||||
``` |
||||
|
||||
PHPT is the end-to-end suite. Build the self-hosted compiler first and pass it |
||||
explicitly to the test runner; using the Zend PHP executable as `--compiler` |
||||
does not test the deployed compiler: |
||||
|
||||
```bash |
||||
PHPX_HOME=/path/to/phpx php bin/tpc.php project.yml --job 2 --no-progress |
||||
php run-tests.php -q -j8 --compiler ./tpc tests/compiler |
||||
``` |
||||
|
||||
Windows packaging requires `PHP_HOME` and `PHPX_HOME`; Linux packaging requires |
||||
UPX; macOS uses `strip` when available. TypePHP rejects 32-bit targets and |
||||
supports common 64-bit CPU architectures, including x86-64 and ARM64. |
||||
Static analysis and the source-derived coverage matrix are separate checks: |
||||
|
||||
```bash |
||||
composer analyse |
||||
php bin/analyze-test-coverage.php |
||||
php bin/analyze-test-coverage.php \ |
||||
--format=markdown --output=build/test-coverage.md --strict |
||||
``` |
||||
|
||||
The coverage tool reports PHP version × feature × positive compilation × |
||||
runtime semantics × negative diagnostics, plus concrete PHP-parser AST nodes. |
||||
It intentionally does not publish a single percentage without an explicit |
||||
denominator. See [Test coverage analyzer](docs/en/TEST_COVERAGE_ANALYZER.md). |
||||
|
||||
GitHub Actions runs PHPUnit and self-hosted PHPT on PHP 8.4 and 8.5. Changes to |
||||
compiler behavior should add a focused PHPUnit test for internal/code-generation |
||||
rules and a PHPT whenever runtime output or diagnostics are observable. |
||||
|
||||
## Documentation |
||||
|
||||
- [Quick Start](docs/en/QUICKSTART.md) — minimal compilation flow |
||||
- [Compilation modes](docs/en/COMPILATION_MODES.md) — `bin`, `ext`, `lib` |
||||
- [Compiler CLI](docs/en/COMPILER_CLI.md) — CLI arguments and project config |
||||
- [Incompatible PHP features](docs/en/INCOMPATIBLE_PHP_FEATURES.md) — current limits |
||||
- [Native types](docs/en/NATIVE_TYPES.md) — native scalar types |
||||
- [High-precision types](docs/en/HIGH_PRECISION_TYPES.md) — BigInt / Decimal / BigFloat |
||||
- [Std containers](docs/en/STD_CONTAINERS.md) — strongly-typed containers |
||||
- [Universal methods](docs/en/UNIVERSAL_METHODS.md) — compile-time method resolution |
||||
- [Compile-time functions](docs/en/COMPILE_TIME_FUNCTIONS.md) — `any()`, `refval()`, `objval()`, … |
||||
- [Mixed C++/PHP](docs/en/MIXED_CPP_PHP.md) — C++/PHP interop |
||||
- [`#[Immutable]`](docs/en/IMMUTABLE.md) — compile-time read-only contracts |
||||
- [`#[ArrayDef]`](docs/en/ARRAY_DEF.md) — typed array-property contracts |
||||
- [Property hooks](docs/en/PROPERTY_HOOKS.md) — PHP 8.4 hook lowering and runtime metadata |
||||
- [Object storage models](docs/en/OBJECT_STORAGE_AND_PASSING_MODELS.md) — Zend object, Box, and Native class boundaries |
||||
- [Generators](docs/en/YIELD_GENERATOR.md) — generator lowering and lifecycle |
||||
- [Test coverage analyzer](docs/en/TEST_COVERAGE_ANALYZER.md) — AST and feature evidence matrix |
||||
- [WASI build](docs/en/WASI_BUILD.md) — WASI targets |
||||
|
||||
## License |
||||
|
||||
TypePHP is licensed under the [GNU General Public License v3.0](LICENSE). |
||||
|
||||
## Community |
||||
|
||||
- Repository: <https://github.com/swoole/typephp> |
||||
- Copyright © 2026 上海识沃网络科技有限公司 (Swoole) |
||||
|
||||
@ -1,52 +0,0 @@ |
||||
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> |
||||
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0"> |
||||
|
||||
<!-- 程序集标识 --> |
||||
<assemblyIdentity |
||||
type="win32" |
||||
name="AotCompiler.App" |
||||
version="1.0.0.0" /> |
||||
|
||||
<!-- 信任信息 / UAC 权限级别 --> |
||||
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v3"> |
||||
<security> |
||||
<requestedPrivileges> |
||||
<!-- |
||||
asInvoker: 以当前用户权限运行(推荐默认值) |
||||
highestAvailable: 请求当前用户能获得的最高权限 |
||||
requireAdministrator: 要求管理员权限 |
||||
--> |
||||
<requestedExecutionLevel level="asInvoker" uiAccess="false" /> |
||||
</requestedPrivileges> |
||||
</security> |
||||
</trustInfo> |
||||
|
||||
<!-- 应用程序兼容性 --> |
||||
<compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1"> |
||||
<application> |
||||
<!-- Windows 10 / 11 --> |
||||
<supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}" /> |
||||
<!-- Windows 8.1 --> |
||||
<supportedOS Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}" /> |
||||
<!-- Windows 8 --> |
||||
<supportedOS Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}" /> |
||||
<!-- Windows 7 --> |
||||
<supportedOS Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}" /> |
||||
</application> |
||||
</compatibility> |
||||
|
||||
<!-- DPI 感知(Windows 10 版本 1607+) --> |
||||
<asmv3:application xmlns:asmv3="urn:schemas-microsoft-com:asm.v3"> |
||||
<asmv3:windowsSettings> |
||||
<!-- |
||||
PerMonitorV2: 逐显示器 DPI 感知 v2(推荐,支持混合模式 DPI 缩放) |
||||
PerMonitor: 逐显示器 DPI 感知 v1 |
||||
System: 系统 DPI 感知 |
||||
None / 不设置: DPI 不感知(系统会自动缩放) |
||||
--> |
||||
<dpiAware xmlns="http://schemas.microsoft.com/SMI/2005/WindowsSettings">true</dpiAware> |
||||
<dpiAwareness xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">PerMonitorV2</dpiAwareness> |
||||
</asmv3:windowsSettings> |
||||
</asmv3:application> |
||||
|
||||
</assembly> |
||||
@ -1,495 +0,0 @@ |
||||
#!/usr/bin/env python3 |
||||
""" |
||||
使用 libclang 解析 C++ 代码并添加头文件路径 |
||||
""" |
||||
|
||||
import clang.cindex |
||||
from clang.cindex import Index, CursorKind, TypeKind, StorageClass |
||||
import json |
||||
import sys |
||||
import os |
||||
from pathlib import Path |
||||
import subprocess |
||||
import argparse |
||||
|
||||
|
||||
class PHPConfigHelper: |
||||
"""PHP 配置辅助类""" |
||||
|
||||
def __init__(self, php_config_path='php-config'): |
||||
self.php_config = php_config_path |
||||
self._check_availability() |
||||
|
||||
def _check_availability(self): |
||||
"""检查 php-config 是否可用""" |
||||
try: |
||||
result = subprocess.run( |
||||
[self.php_config, '--version'], |
||||
capture_output=True, |
||||
text=True, |
||||
check=True |
||||
) |
||||
print(f"✓ 找到 PHP {result.stdout.strip()}") |
||||
except (FileNotFoundError, subprocess.CalledProcessError) as e: |
||||
print(f"警告: php-config 不可用: {e}") |
||||
# 尝试使用常见路径 |
||||
common_paths = [ |
||||
'/usr/bin/php-config', |
||||
'/usr/local/bin/php-config', |
||||
'/opt/php/bin/php-config' |
||||
] |
||||
for path in common_paths: |
||||
if os.path.exists(path): |
||||
self.php_config = path |
||||
try: |
||||
result = subprocess.run( |
||||
[self.php_config, '--version'], |
||||
capture_output=True, |
||||
text=True, |
||||
check=True |
||||
) |
||||
print(f"✓ 找到 PHP {result.stdout.strip()} 在 {path}") |
||||
return |
||||
except (FileNotFoundError, subprocess.CalledProcessError): |
||||
continue |
||||
print("警告: php-config 在任何常见路径都不可用,使用默认路径") |
||||
|
||||
def get_includes(self): |
||||
""" |
||||
获取 include 路径列表 |
||||
|
||||
Returns: |
||||
list: 头文件路径列表(不带 -I 前缀) |
||||
""" |
||||
try: |
||||
result = subprocess.run( |
||||
[self.php_config, '--includes'], |
||||
capture_output=True, |
||||
text=True, |
||||
check=True |
||||
) |
||||
|
||||
# 解析输出: "-I/path1 -I/path2" -> ['/path1', '/path2'] |
||||
includes = [] |
||||
for flag in result.stdout.strip().split(): |
||||
if flag.startswith('-I'): |
||||
includes.append(flag[2:]) |
||||
|
||||
return includes |
||||
except (subprocess.CalledProcessError, FileNotFoundError): |
||||
print("警告: 无法获取 PHP includes,使用默认路径") |
||||
return ['/usr/include/php', '/usr/include/php/20210902'] # 默认路径 |
||||
|
||||
def get_include_dir(self): |
||||
"""获取主 include 目录""" |
||||
try: |
||||
result = subprocess.run( |
||||
[self.php_config, '--include-dir'], |
||||
capture_output=True, |
||||
text=True, |
||||
check=True |
||||
) |
||||
return result.stdout.strip() |
||||
except (subprocess.CalledProcessError, FileNotFoundError): |
||||
return '/usr/include/php' |
||||
|
||||
def get_extension_dir(self): |
||||
"""获取扩展目录""" |
||||
try: |
||||
result = subprocess.run( |
||||
[self.php_config, '--extension-dir'], |
||||
capture_output=True, |
||||
text=True, |
||||
check=True |
||||
) |
||||
return result.stdout.strip() |
||||
except (subprocess.CalledProcessError, FileNotFoundError): |
||||
return '/usr/lib/php' |
||||
|
||||
def get_version(self): |
||||
"""获取 PHP 版本""" |
||||
try: |
||||
result = subprocess.run( |
||||
[self.php_config, '--version'], |
||||
capture_output=True, |
||||
text=True, |
||||
check=True |
||||
) |
||||
return result.stdout.strip() |
||||
except (subprocess.CalledProcessError, FileNotFoundError): |
||||
return 'unknown' |
||||
|
||||
def get_php_binary(self): |
||||
"""获取 PHP 二进制路径""" |
||||
try: |
||||
result = subprocess.run( |
||||
[self.php_config, '--php-binary'], |
||||
capture_output=True, |
||||
text=True, |
||||
check=True |
||||
) |
||||
return result.stdout.strip() |
||||
except (subprocess.CalledProcessError, FileNotFoundError): |
||||
return 'php' |
||||
|
||||
def get_configure_options(self): |
||||
"""获取配置选项""" |
||||
try: |
||||
result = subprocess.run( |
||||
[self.php_config, '--configure-options'], |
||||
capture_output=True, |
||||
text=True, |
||||
check=True |
||||
) |
||||
return result.stdout.strip() |
||||
except (subprocess.CalledProcessError, FileNotFoundError): |
||||
return '' |
||||
|
||||
def get_all_info(self): |
||||
"""获取所有配置信息""" |
||||
return { |
||||
'version': self.get_version(), |
||||
'includes': self.get_includes(), |
||||
'include_dir': self.get_include_dir(), |
||||
'extension_dir': self.get_extension_dir(), |
||||
'php_binary': self.get_php_binary(), |
||||
'configure_options': self.get_configure_options(), |
||||
} |
||||
|
||||
|
||||
class ClangParser: |
||||
def __init__(self, libclang_path=None): |
||||
""" |
||||
初始化 Clang 解析器 |
||||
|
||||
Args: |
||||
libclang_path: libclang 库的路径(可选) |
||||
""" |
||||
if libclang_path: |
||||
try: |
||||
clang.cindex.Config.set_library_file(libclang_path) |
||||
except Exception as e: |
||||
print(f"警告: 无法设置 libclang 路径 {libclang_path}: {e}") |
||||
print("尝试使用默认路径...") |
||||
|
||||
try: |
||||
self.index = Index.create() |
||||
except Exception as e: |
||||
print(f"错误: 无法创建 Clang 索引: {e}") |
||||
print("请确保已安装 python3-clang 和 clang 库") |
||||
raise |
||||
|
||||
def parse_file(self, filename, include_paths=None, defines=None, |
||||
compiler_args=None, language='c++'): |
||||
""" |
||||
解析 C++ 文件 |
||||
|
||||
Args: |
||||
filename: 要解析的文件路径 |
||||
include_paths: 头文件搜索路径列表 |
||||
defines: 宏定义列表 ['MACRO=value', 'DEBUG'] |
||||
compiler_args: 额外的编译器参数 |
||||
language: 语言类型 ('c', 'c++', 'objective-c') |
||||
|
||||
Returns: |
||||
TranslationUnit 对象 |
||||
""" |
||||
if not os.path.exists(filename): |
||||
raise FileNotFoundError(f"文件不存在: {filename}") |
||||
|
||||
args = [] |
||||
|
||||
# 1. 设置语言标准 |
||||
if language == 'c++': |
||||
args.extend([ |
||||
'-x', 'c++', |
||||
'-std=c++14', # 更标准的 C++ 版本 |
||||
]) |
||||
elif language == 'c': |
||||
args.extend(['-x', 'c', '-std=c11']) |
||||
|
||||
# 2. 添加头文件搜索路径 |
||||
if include_paths: |
||||
for path in include_paths: |
||||
if os.path.exists(path): # 检查路径是否存在 |
||||
args.append(f'-I{path}') |
||||
else: |
||||
print(f"警告: 包含路径不存在: {path}") |
||||
|
||||
# 3. 添加宏定义 |
||||
if defines: |
||||
for define in defines: |
||||
args.append(f'-D{define}') |
||||
|
||||
# 4. 添加额外的编译器参数 |
||||
if compiler_args: |
||||
args.extend(compiler_args) |
||||
|
||||
# 5. 常用的编译选项 |
||||
args.extend([ |
||||
'-Wno-pragma-once-outside-header', # 忽略警告 |
||||
'-ferror-limit=0', # 不限制错误数量 |
||||
'-fno-delayed-template-parsing', # 避免某些 C++ 模板解析问题 |
||||
'-w', # 禁用所有警告以减少输出 |
||||
]) |
||||
|
||||
print(f"编译参数: {' '.join(args)}") |
||||
|
||||
# 解析文件 |
||||
try: |
||||
tu = self.index.parse( |
||||
filename, |
||||
args=args, |
||||
options=clang.cindex.TranslationUnit.PARSE_DETAILED_PROCESSING_RECORD |
||||
) |
||||
except Exception as e: |
||||
print(f"解析文件时出错: {e}") |
||||
print("尝试使用最小参数集...") |
||||
# 尝试使用最小参数集 |
||||
minimal_args = ['-x', 'c++', '-std=c++14', '-w'] |
||||
if include_paths: |
||||
for path in include_paths: |
||||
if os.path.exists(path): |
||||
minimal_args.append(f'-I{path}') |
||||
try: |
||||
tu = self.index.parse( |
||||
filename, |
||||
args=minimal_args, |
||||
options=clang.cindex.TranslationUnit.PARSE_DETAILED_PROCESSING_RECORD |
||||
) |
||||
print("使用最小参数集成功解析") |
||||
except Exception as e2: |
||||
print(f"使用最小参数集也失败: {e2}") |
||||
print("尝试解析不包含头文件的简化版本...") |
||||
# 创建一个临时文件,移除头文件包含行 |
||||
temp_filename = filename + ".tmp" |
||||
with open(filename, 'r') as original: |
||||
lines = original.readlines() |
||||
|
||||
# 移除 #include 行 |
||||
filtered_lines = [line for line in lines if not line.strip().startswith('#include')] |
||||
|
||||
with open(temp_filename, 'w') as temp: |
||||
temp.writelines(filtered_lines) |
||||
|
||||
try: |
||||
tu = self.index.parse( |
||||
temp_filename, |
||||
args=minimal_args, |
||||
options=clang.cindex.TranslationUnit.PARSE_DETAILED_PROCESSING_RECORD |
||||
) |
||||
print("解析简化版本成功") |
||||
# 清理临时文件 |
||||
os.remove(temp_filename) |
||||
except Exception as e3: |
||||
print(f"简化版本也失败: {e3}") |
||||
# 清理临时文件 |
||||
if os.path.exists(temp_filename): |
||||
os.remove(temp_filename) |
||||
raise |
||||
|
||||
# 检查诊断信息 |
||||
if tu.diagnostics: |
||||
print(f"\n诊断信息 ({len(tu.diagnostics)} 个):") |
||||
error_count = 0 |
||||
warning_count = 0 |
||||
|
||||
for diag in tu.diagnostics: |
||||
if diag.severity >= 3: # 错误级别 |
||||
error_count += 1 |
||||
else: # 警告级别 |
||||
warning_count += 1 |
||||
|
||||
print(f"错误: {error_count}, 警告: {warning_count}") |
||||
|
||||
# 只显示前几个诊断信息,避免输出过多 |
||||
for i, diag in enumerate(tu.diagnostics): |
||||
if i >= 5: # 只显示前5个 |
||||
print("... 还有更多诊断信息") |
||||
break |
||||
print(f" [{diag.severity}] {diag.spelling}") |
||||
if diag.location.file: |
||||
print(f" at {diag.location.file.name}:{diag.location.line}") |
||||
|
||||
return tu |
||||
|
||||
def extract_functions(self, tu, name_prefixes=None): |
||||
""" |
||||
提取函数定义 |
||||
|
||||
Args: |
||||
tu: TranslationUnit 对象 |
||||
name_prefixes: 函数名前缀过滤列表 |
||||
|
||||
Returns: |
||||
函数信息列表 |
||||
""" |
||||
functions = [] |
||||
|
||||
def visit_node(node, depth=0): |
||||
# 只处理函数声明/定义 |
||||
if node.kind == CursorKind.FUNCTION_DECL: |
||||
try: |
||||
func_info = self.parse_function(node) |
||||
|
||||
# 过滤函数名 |
||||
if name_prefixes: |
||||
if any(func_info['name'].startswith(prefix) |
||||
for prefix in name_prefixes): |
||||
functions.append(func_info) |
||||
else: |
||||
functions.append(func_info) |
||||
except Exception as e: |
||||
print(f"解析函数时出错: {e}") |
||||
|
||||
# 递归访问子节点 |
||||
for child in node.get_children(): |
||||
visit_node(child, depth + 1) |
||||
|
||||
visit_node(tu.cursor) |
||||
return functions |
||||
|
||||
def parse_function(self, cursor): |
||||
""" |
||||
解析函数详细信息 |
||||
""" |
||||
# 检查方法是否存在 |
||||
def safe_call(method, default_value=None): |
||||
try: |
||||
return method() |
||||
except AttributeError: |
||||
return default_value |
||||
|
||||
# 基本信息 |
||||
func_info = { |
||||
'name': cursor.spelling, |
||||
'displayName': cursor.displayname, |
||||
'mangledName': cursor.mangled_name, |
||||
'returnType': cursor.result_type.spelling, |
||||
'isStatic': cursor.storage_class == StorageClass.STATIC, |
||||
'isInline': safe_call(lambda: cursor.is_inline_function(), False), |
||||
'isVirtual': safe_call(lambda: cursor.is_virtual_method(), False), |
||||
'isConst': safe_call(lambda: cursor.is_const_method(), False), |
||||
'location': { |
||||
'file': str(cursor.location.file) if cursor.location.file else None, |
||||
'line': cursor.location.line, |
||||
'column': cursor.location.column, |
||||
}, |
||||
'parameters': [], |
||||
'namespaces': self.get_namespaces(cursor), |
||||
} |
||||
|
||||
# 解析参数 |
||||
for arg in cursor.get_arguments(): |
||||
param_info = { |
||||
'name': arg.spelling or f'arg{len(func_info["parameters"])}', |
||||
'type': arg.type.spelling, |
||||
'canonicalType': arg.type.get_canonical().spelling, |
||||
} |
||||
|
||||
# 检查是否有默认值 |
||||
try: |
||||
for token in arg.get_tokens(): |
||||
if token.spelling == '=': |
||||
# 有默认值 |
||||
param_info['hasDefault'] = True |
||||
break |
||||
except: |
||||
# 如果无法获取 tokens,跳过默认值检查 |
||||
pass |
||||
|
||||
func_info['parameters'].append(param_info) |
||||
|
||||
return func_info |
||||
|
||||
def get_namespaces(self, cursor): |
||||
""" |
||||
获取函数所在的命名空间 |
||||
""" |
||||
namespaces = [] |
||||
parent = cursor.semantic_parent |
||||
|
||||
while parent and parent.kind != CursorKind.TRANSLATION_UNIT: |
||||
if parent.kind == CursorKind.NAMESPACE: |
||||
namespaces.insert(0, parent.spelling) |
||||
parent = parent.semantic_parent |
||||
|
||||
return namespaces |
||||
|
||||
|
||||
def main(): |
||||
parser = argparse.ArgumentParser(description='使用 libclang 解析 C++ 代码并提取函数信息') |
||||
parser.add_argument('filename', help='要解析的 C++ 文件路径') |
||||
parser.add_argument('--libclang-path', help='libclang 库路径') |
||||
parser.add_argument('--include-paths', nargs='*', help='额外的包含路径') |
||||
parser.add_argument('--function-prefixes', nargs='*', help='函数名前缀过滤器') |
||||
|
||||
args = parser.parse_args() |
||||
|
||||
if not os.path.exists(args.filename): |
||||
print(f"错误: 文件不存在: {args.filename}") |
||||
sys.exit(1) |
||||
|
||||
try: |
||||
# 创建解析器 |
||||
parser_obj = ClangParser(libclang_path=args.libclang_path) |
||||
|
||||
# 配置头文件路径 |
||||
include_paths = args.include_paths or [ |
||||
"/usr/include/linux", |
||||
"/home/swoole/workspace/projects/phpx/include" |
||||
] |
||||
|
||||
# 尝试获取 PHP 配置的头文件路径 |
||||
try: |
||||
php_config = PHPConfigHelper() |
||||
php_includes = php_config.get_includes() |
||||
include_paths.extend(php_includes) |
||||
except Exception as e: |
||||
print(f"警告: 无法获取 PHP 配置: {e}") |
||||
print("继续使用默认路径...") |
||||
|
||||
# 配置宏定义 |
||||
defines = [ |
||||
'HAVE_CONFIG_H', |
||||
'ZEND_ENABLE_STATIC_TSRMLS_CACHE=1', |
||||
] |
||||
|
||||
# 额外的编译器参数 |
||||
compiler_args = [ |
||||
'-fparse-all-comments', # 解析所有注释 |
||||
'-Wno-unknown-pragmas', |
||||
] |
||||
|
||||
# 解析文件 |
||||
tu = parser_obj.parse_file( |
||||
args.filename, |
||||
include_paths=include_paths, |
||||
defines=defines, |
||||
compiler_args=compiler_args, |
||||
language='c++' |
||||
) |
||||
|
||||
# 提取函数 |
||||
name_prefixes = args.function_prefixes or None |
||||
functions = parser_obj.extract_functions(tu, name_prefixes=name_prefixes) |
||||
|
||||
# 输出结果 |
||||
output = { |
||||
'file': args.filename, |
||||
'functions': functions, |
||||
'total': len(functions), |
||||
} |
||||
|
||||
print(json.dumps(output, indent=2, ensure_ascii=False)) |
||||
|
||||
except clang.cindex.TranslationUnitLoadError as e: |
||||
print(f"翻译单元加载错误: {e}") |
||||
print("这通常意味着 C++ 代码包含语法错误或缺少必要的头文件") |
||||
sys.exit(1) |
||||
except Exception as e: |
||||
print(f"错误: {e}") |
||||
sys.exit(1) |
||||
|
||||
if __name__ == '__main__': |
||||
main() |
||||
@ -0,0 +1,22 @@ |
||||
# TypePHP benchmarks |
||||
|
||||
This directory contains repeatable performance workloads used to guide and |
||||
verify compiler/runtime optimizations. Benchmark results depend on the CPU, |
||||
PHP build, compiler, and system load, so compare PHP and TypePHP on the same |
||||
machine instead of committing absolute timing expectations. |
||||
|
||||
- `bench.php` and `micro_bench.php` are the original general workloads moved |
||||
from `examples/`. |
||||
- `property-access/` builds and compares dynamic/static property access under |
||||
Zend PHP and TypePHP. |
||||
|
||||
Run the property benchmark from the repository root: |
||||
|
||||
```bash |
||||
php benchmark/property-access/run.php |
||||
``` |
||||
|
||||
The property benchmark builds the generated application with `-O3` and LTO. |
||||
For meaningful results, link it against a Release build of PHPX as well; a |
||||
Debug/`-O0` `libphpx` makes property helper calls several times slower and is |
||||
not representative of a release package. |
||||
@ -0,0 +1,3 @@ |
||||
/build/ |
||||
/property_access |
||||
/*.rsp |
||||
@ -0,0 +1,16 @@ |
||||
# Dynamic property benchmark |
||||
|
||||
This benchmark compares the same dynamic and static property operations under |
||||
Zend PHP and a TypePHP `-O3` + LTO binary. Each metric is the best of seven |
||||
rounds after three warm-up rounds and is reported in nanoseconds per property |
||||
access. |
||||
|
||||
Run it from the repository root: |
||||
|
||||
```bash |
||||
php benchmark/property-access/run.php |
||||
``` |
||||
|
||||
To reuse an existing binary, add `--skip-build`. For local regression checks, |
||||
`--max-ratio=1.5` exits unsuccessfully when a dynamic read or write takes more |
||||
than 1.5 times the corresponding Zend PHP result. |
||||
@ -0,0 +1,154 @@ |
||||
<?php |
||||
|
||||
declare(strict_types=1); |
||||
|
||||
final class DynamicPropertyEntity |
||||
{ |
||||
public int $first = 0; |
||||
public int $second = 0; |
||||
public int $third = 0; |
||||
public int $fourth = 0; |
||||
public int $fifth = 0; |
||||
|
||||
public function hydrate(array $data): void |
||||
{ |
||||
foreach ($data as $property => $value) { |
||||
$this->$property = $value; |
||||
} |
||||
} |
||||
|
||||
public function sum(array $properties): int |
||||
{ |
||||
$sum = 0; |
||||
foreach ($properties as $property) { |
||||
$sum += $this->$property; |
||||
} |
||||
return $sum; |
||||
} |
||||
} |
||||
|
||||
final class StaticPropertyEntity |
||||
{ |
||||
public int $first = 0; |
||||
public int $second = 0; |
||||
public int $third = 0; |
||||
public int $fourth = 0; |
||||
public int $fifth = 0; |
||||
|
||||
public function hydrate(array $data): void |
||||
{ |
||||
$this->first = $data['first']; |
||||
$this->second = $data['second']; |
||||
$this->third = $data['third']; |
||||
$this->fourth = $data['fourth']; |
||||
$this->fifth = $data['fifth']; |
||||
} |
||||
|
||||
public function sum(): int |
||||
{ |
||||
return $this->first + $this->second + $this->third + $this->fourth + $this->fifth; |
||||
} |
||||
} |
||||
|
||||
function runDynamicWrite(DynamicPropertyEntity $entity, array $data, int $iterations): int |
||||
{ |
||||
for ($i = 0; $i < $iterations; $i++) { |
||||
$entity->hydrate($data); |
||||
} |
||||
return $entity->first; |
||||
} |
||||
|
||||
function runStaticWrite(StaticPropertyEntity $entity, array $data, int $iterations): int |
||||
{ |
||||
for ($i = 0; $i < $iterations; $i++) { |
||||
$entity->hydrate($data); |
||||
} |
||||
return $entity->first; |
||||
} |
||||
|
||||
function runDynamicRead(DynamicPropertyEntity $entity, array $properties, int $iterations): int |
||||
{ |
||||
$sum = 0; |
||||
for ($i = 0; $i < $iterations; $i++) { |
||||
$sum += $entity->sum($properties); |
||||
} |
||||
return $sum; |
||||
} |
||||
|
||||
function runStaticRead(StaticPropertyEntity $entity, int $iterations): int |
||||
{ |
||||
$sum = 0; |
||||
for ($i = 0; $i < $iterations; $i++) { |
||||
$sum += $entity->sum(); |
||||
} |
||||
return $sum; |
||||
} |
||||
|
||||
function measure(callable $callback, int $operations): float |
||||
{ |
||||
global $benchmarkSink; |
||||
for ($warmup = 0; $warmup < 3; $warmup++) { |
||||
$benchmarkSink += $callback(); |
||||
} |
||||
|
||||
$best = 1.0e30; |
||||
for ($round = 0; $round < 7; $round++) { |
||||
$start = hrtime(true); |
||||
$result = $callback(); |
||||
$elapsed = hrtime(true) - $start; |
||||
$benchmarkSink += $result; |
||||
if ($elapsed < $best) { |
||||
$best = $elapsed; |
||||
} |
||||
} |
||||
return $best / $operations; |
||||
} |
||||
|
||||
function main(): void |
||||
{ |
||||
global $benchmarkSink; |
||||
$benchmarkSink = 0; |
||||
$iterations = 200000; |
||||
$data = [ |
||||
'first' => 1, |
||||
'second' => 2, |
||||
'third' => 3, |
||||
'fourth' => 4, |
||||
'fifth' => 5, |
||||
]; |
||||
$properties = ['first', 'second', 'third', 'fourth', 'fifth']; |
||||
$dynamic = new DynamicPropertyEntity(); |
||||
$static = new StaticPropertyEntity(); |
||||
$operations = $iterations * 5; |
||||
|
||||
$dynamicWrite = measure( |
||||
function () use ($dynamic, $data, $iterations): int { |
||||
return runDynamicWrite($dynamic, $data, $iterations); |
||||
}, |
||||
$operations, |
||||
); |
||||
$staticWrite = measure( |
||||
function () use ($static, $data, $iterations): int { |
||||
return runStaticWrite($static, $data, $iterations); |
||||
}, |
||||
$operations, |
||||
); |
||||
$dynamicRead = measure( |
||||
function () use ($dynamic, $properties, $iterations): int { |
||||
return runDynamicRead($dynamic, $properties, $iterations); |
||||
}, |
||||
$operations, |
||||
); |
||||
$staticRead = measure( |
||||
function () use ($static, $iterations): int { |
||||
return runStaticRead($static, $iterations); |
||||
}, |
||||
$operations, |
||||
); |
||||
|
||||
printf("dynamic_write_ns=%.3f\n", $dynamicWrite); |
||||
printf("static_write_ns=%.3f\n", $staticWrite); |
||||
printf("dynamic_read_ns=%.3f\n", $dynamicRead); |
||||
printf("static_read_ns=%.3f\n", $staticRead); |
||||
echo 'checksum=', $benchmarkSink + $dynamic->sum($properties) + $static->sum(), "\n"; |
||||
} |
||||
@ -0,0 +1,8 @@ |
||||
name: property_access_benchmark |
||||
mode: bin |
||||
optimize: 3 |
||||
lto: true |
||||
build-dir: build |
||||
output: property_access |
||||
sources: |
||||
- benchmark.php |
||||
@ -0,0 +1,123 @@ |
||||
<?php |
||||
|
||||
declare(strict_types=1); |
||||
|
||||
$root = dirname(__DIR__, 2); |
||||
$source = __DIR__ . '/benchmark.php'; |
||||
$project = __DIR__ . '/project.yml'; |
||||
$binary = __DIR__ . '/property_access'; |
||||
$skipBuild = in_array('--skip-build', $argv, true); |
||||
$maximumRatio = null; |
||||
foreach ($argv as $argument) { |
||||
if (str_starts_with($argument, '--max-ratio=')) { |
||||
$maximumRatio = (float) substr($argument, strlen('--max-ratio=')); |
||||
} |
||||
} |
||||
|
||||
/** |
||||
* @param list<string> $command |
||||
* @param array<string, string>|null $environment |
||||
*/ |
||||
function runCommand(array $command, string $cwd, bool $capture, ?array $environment = null): string |
||||
{ |
||||
$stdout = $capture ? ['pipe', 'w'] : STDOUT; |
||||
$stderr = $capture ? ['pipe', 'w'] : STDERR; |
||||
$process = proc_open( |
||||
$command, |
||||
[STDIN, $stdout, $stderr], |
||||
$pipes, |
||||
$cwd, |
||||
$environment, |
||||
['bypass_shell' => true], |
||||
); |
||||
if (!is_resource($process)) { |
||||
throw new RuntimeException('Failed to start: ' . implode(' ', $command)); |
||||
} |
||||
|
||||
$output = ''; |
||||
$error = ''; |
||||
if ($capture) { |
||||
$output = stream_get_contents($pipes[1]); |
||||
$error = stream_get_contents($pipes[2]); |
||||
fclose($pipes[1]); |
||||
fclose($pipes[2]); |
||||
} |
||||
$status = proc_close($process); |
||||
if ($status !== 0) { |
||||
throw new RuntimeException( |
||||
'Command failed (' . $status . '): ' . implode(' ', $command) . "\n" . $output . $error, |
||||
); |
||||
} |
||||
return $output; |
||||
} |
||||
|
||||
/** @return array<string, float> */ |
||||
function parseResults(string $output): array |
||||
{ |
||||
$results = []; |
||||
foreach (explode("\n", trim($output)) as $line) { |
||||
if (!str_contains($line, '=')) { |
||||
continue; |
||||
} |
||||
[$name, $value] = explode('=', $line, 2); |
||||
if ($name !== 'checksum') { |
||||
$results[$name] = (float) $value; |
||||
} |
||||
} |
||||
return $results; |
||||
} |
||||
|
||||
if (!$skipBuild) { |
||||
runCommand([ |
||||
PHP_BINARY, |
||||
$root . '/bin/tpc.php', |
||||
$project, |
||||
'-j', |
||||
'8', |
||||
'--no-color', |
||||
'--no-progress', |
||||
], $root, false); |
||||
} |
||||
if (!is_file($binary)) { |
||||
throw new RuntimeException('Benchmark binary does not exist: ' . $binary); |
||||
} |
||||
|
||||
$php = parseResults(runCommand([ |
||||
PHP_BINARY, |
||||
'-d', |
||||
'opcache.enable_cli=0', |
||||
'-r', |
||||
'require ' . var_export($source, true) . '; main();', |
||||
], $root, true)); |
||||
$typephpEnvironment = null; |
||||
if (PHP_OS_FAMILY !== 'Windows') { |
||||
$phpxHome = getenv('PHPX_HOME'); |
||||
if (!is_string($phpxHome) || $phpxHome === '') { |
||||
$phpxHome = $root . '/vendor/swoole/phpx'; |
||||
} |
||||
$typephpEnvironment = getenv(); |
||||
$loaderVariable = PHP_OS_FAMILY === 'Darwin' ? 'DYLD_LIBRARY_PATH' : 'LD_LIBRARY_PATH'; |
||||
$existingPath = $typephpEnvironment[$loaderVariable] ?? ''; |
||||
$typephpEnvironment[$loaderVariable] = $phpxHome . '/lib' |
||||
. ($existingPath === '' ? '' : PATH_SEPARATOR . $existingPath); |
||||
} |
||||
$typephp = parseResults(runCommand([$binary], $root, true, $typephpEnvironment)); |
||||
|
||||
echo "Metric PHP ns/op TypePHP ns/op TypePHP/PHP\n"; |
||||
echo "------------------------------------------------------------\n"; |
||||
$failed = false; |
||||
foreach (['dynamic_write_ns', 'dynamic_read_ns', 'static_write_ns', 'static_read_ns'] as $metric) { |
||||
if (!isset($php[$metric], $typephp[$metric])) { |
||||
throw new RuntimeException('Missing benchmark metric: ' . $metric); |
||||
} |
||||
$ratio = $typephp[$metric] / $php[$metric]; |
||||
printf("%-22s %10.2f %14.2f %12.2fx\n", $metric, $php[$metric], $typephp[$metric], $ratio); |
||||
if ($maximumRatio !== null && str_starts_with($metric, 'dynamic_') && $ratio > $maximumRatio) { |
||||
$failed = true; |
||||
} |
||||
} |
||||
|
||||
if ($failed) { |
||||
fwrite(STDERR, "Dynamic property ratio exceeded --max-ratio={$maximumRatio}\n"); |
||||
exit(1); |
||||
} |
||||
@ -0,0 +1,146 @@ |
||||
#!/usr/bin/env php |
||||
<?php |
||||
/** |
||||
* This file is part of Swoole-Compiler(AOT). |
||||
* |
||||
* @link https://www.swoole.com/ |
||||
* @contact service@swoole.com |
||||
*/ |
||||
|
||||
declare(strict_types=1); |
||||
|
||||
use TypePhp\Testing\TestCoverageAnalyzer; |
||||
|
||||
require __DIR__ . '/bootstrap.php'; |
||||
|
||||
$format = 'summary'; |
||||
$output = null; |
||||
$includePhpUnit = true; |
||||
$strict = false; |
||||
$phpVersions = ['8.4', '8.5']; |
||||
$paths = []; |
||||
|
||||
foreach (array_slice($argv, 1) as $argument) { |
||||
if ($argument === '--help' || $argument === '-h') { |
||||
printUsage($argv[0]); |
||||
exit(0); |
||||
} |
||||
if ($argument === '--no-phpunit') { |
||||
$includePhpUnit = false; |
||||
continue; |
||||
} |
||||
if ($argument === '--strict') { |
||||
$strict = true; |
||||
continue; |
||||
} |
||||
if (str_starts_with($argument, '--format=')) { |
||||
$format = substr($argument, strlen('--format=')); |
||||
continue; |
||||
} |
||||
if (str_starts_with($argument, '--output=')) { |
||||
$output = substr($argument, strlen('--output=')); |
||||
continue; |
||||
} |
||||
if (str_starts_with($argument, '--php-versions=')) { |
||||
$phpVersions = array_values(array_filter(array_map('trim', explode(',', substr($argument, strlen('--php-versions=')))))); |
||||
continue; |
||||
} |
||||
if (str_starts_with($argument, '-')) { |
||||
fwrite(STDERR, 'Unknown option: ' . $argument . PHP_EOL); |
||||
exit(2); |
||||
} |
||||
$paths[] = $argument; |
||||
} |
||||
|
||||
if (!in_array($format, ['summary', 'json', 'markdown'], true)) { |
||||
fwrite(STDERR, 'Invalid format. Expected summary, json or markdown.' . PHP_EOL); |
||||
exit(2); |
||||
} |
||||
if ($phpVersions === []) { |
||||
fwrite(STDERR, 'At least one target PHP version is required.' . PHP_EOL); |
||||
exit(2); |
||||
} |
||||
foreach ($phpVersions as $version) { |
||||
if (!preg_match('/^\d+\.\d+$/', $version)) { |
||||
fwrite(STDERR, 'Invalid PHP version: ' . $version . PHP_EOL); |
||||
exit(2); |
||||
} |
||||
} |
||||
if ($paths === []) { |
||||
$paths = ['tests/compiler']; |
||||
} |
||||
|
||||
try { |
||||
$analyzer = new TestCoverageAnalyzer(TYPEPHP_ROOT_PATH, $phpVersions); |
||||
$report = $analyzer->analyze( |
||||
$paths, |
||||
$includePhpUnit ? TYPEPHP_ROOT_PATH . '/phpunit/src' : null, |
||||
$includePhpUnit ? TYPEPHP_ROOT_PATH . '/phpunit/code' : null, |
||||
); |
||||
} catch (Throwable $error) { |
||||
fwrite(STDERR, 'Coverage analysis failed: ' . $error->getMessage() . PHP_EOL); |
||||
exit(1); |
||||
} |
||||
|
||||
$rendered = match ($format) { |
||||
'summary' => $analyzer->renderSummary($report), |
||||
'markdown' => $analyzer->renderMarkdown($report), |
||||
'json' => json_encode($report, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE | JSON_THROW_ON_ERROR) . PHP_EOL, |
||||
}; |
||||
|
||||
if ($output === null) { |
||||
echo $rendered; |
||||
} else { |
||||
$outputPath = isAbsolutePath($output) ? $output : TYPEPHP_ROOT_PATH . DIRECTORY_SEPARATOR . $output; |
||||
$directory = dirname($outputPath); |
||||
if (!is_dir($directory) && !mkdir($directory, 0777, true) && !is_dir($directory)) { |
||||
fwrite(STDERR, 'Unable to create output directory: ' . $directory . PHP_EOL); |
||||
exit(1); |
||||
} |
||||
if (file_put_contents($outputPath, $rendered) === false) { |
||||
fwrite(STDERR, 'Unable to write report: ' . $outputPath . PHP_EOL); |
||||
exit(1); |
||||
} |
||||
echo 'Wrote ', $format, ' coverage report: ', relativePath(TYPEPHP_ROOT_PATH, $outputPath), PHP_EOL; |
||||
} |
||||
|
||||
if ($strict && ($report['parse_errors'] !== [] || $report['unresolved_phpunit_fixtures'] !== [])) { |
||||
exit(1); |
||||
} |
||||
|
||||
function printUsage(string $script): void |
||||
{ |
||||
echo <<<USAGE |
||||
Usage: |
||||
php {$script} [options] [PHPT path ...] |
||||
|
||||
Options: |
||||
--format=summary|json|markdown Output format (default: summary) |
||||
--output=<file> Write the report to a file |
||||
--php-versions=8.4,8.5 Target PHP version columns |
||||
--no-phpunit Do not scan PHPUnit compiler fixtures |
||||
--strict Fail on parse issues or unresolved fixture links |
||||
-h, --help Show this help |
||||
|
||||
Examples: |
||||
php {$script} |
||||
php {$script} --format=markdown --output=build/test-coverage.md |
||||
php {$script} --format=json tests/compiler/type_decl tests/compiler/basic |
||||
|
||||
The tool reports separate, explicitly denominated AST-node, positive compile, |
||||
runtime semantic and negative diagnostic coverage. It never emits a combined |
||||
overall percentage. |
||||
|
||||
USAGE; |
||||
} |
||||
|
||||
function isAbsolutePath(string $path): bool |
||||
{ |
||||
return $path !== '' && ($path[0] === '/' || preg_match('/^[A-Za-z]:[\\\\\/]/', $path) === 1); |
||||
} |
||||
|
||||
function relativePath(string $root, string $path): string |
||||
{ |
||||
$prefix = rtrim($root, DIRECTORY_SEPARATOR) . DIRECTORY_SEPARATOR; |
||||
return str_starts_with($path, $prefix) ? substr($path, strlen($prefix)) : $path; |
||||
} |
||||
@ -1,6 +1,17 @@ |
||||
<?php |
||||
define("ROOT_PATH", dirname(__DIR__)); |
||||
define('DEBUG', true); |
||||
/** |
||||
* This file is part of TypePHP(AOT). |
||||
* |
||||
* @link https://www.swoole.com/aot/ |
||||
* @contact service@swoole.com |
||||
*/ |
||||
|
||||
require ROOT_PATH . '/vendor/autoload.php'; |
||||
require ROOT_PATH . '/src/functions.php'; |
||||
define('TYPEPHP_ROOT_PATH', dirname(__DIR__)); |
||||
define('TYPEPHP_DEBUG', true); |
||||
|
||||
// Composer bin proxies provide the consuming project's autoloader. A source |
||||
// checkout and a packaged compiler keep their own autoloader below TYPEPHP_ROOT_PATH. |
||||
$autoloadPath = $GLOBALS['_composer_autoload_path'] ?? TYPEPHP_ROOT_PATH . '/vendor/autoload.php'; |
||||
unset($GLOBALS['_composer_autoload_path']); |
||||
require $autoloadPath; |
||||
unset($autoloadPath); |
||||
|
||||
@ -1,42 +0,0 @@ |
||||
<?php |
||||
$dir = '/home/htf/workspace/c/php-8.0.12/ext/standard/'; |
||||
|
||||
$outDir = __DIR__ . '/../config/'; |
||||
|
||||
$funcs = []; |
||||
$files = glob($dir . '*.c'); |
||||
$index = 0; |
||||
foreach ($files as $file) { |
||||
$content = file_get_contents($file); |
||||
preg_match_all('/PHP_FUNCTION\(([a-z0-9_]+)\)/i', $content, $match); |
||||
if (empty($match[1])) { |
||||
continue; |
||||
} |
||||
foreach ($match[1] as $fn) { |
||||
$funcs[] = $fn; |
||||
} |
||||
} |
||||
|
||||
shuffle($funcs); |
||||
|
||||
$_funcs = []; |
||||
foreach ($funcs as $fn) { |
||||
$_funcs['fn_' . random_int(100000, 999999) . str_pad($index++, 3, '0') . random_int(100, 999)] = $fn; |
||||
} |
||||
|
||||
function dumpVar($file, $var) |
||||
{ |
||||
file_put_contents($file, "<?php\nreturn " . var_export($var, 1) . ";\n");
|
||||
} |
||||
|
||||
dumpVar($outDir . 'functions.php', $_funcs); |
||||
|
||||
$constants = get_defined_constants(); |
||||
$ignore_constants = array_flip(require __DIR__ . '/ignore_constants.php'); |
||||
foreach ($constants as $k => $v) { |
||||
if (isset($ignore_constants[$k])) { |
||||
unset($constants[$k]); |
||||
} |
||||
} |
||||
|
||||
dumpVar($outDir . 'constants.php', $constants); |
||||
@ -1,40 +0,0 @@ |
||||
<?php |
||||
return [ |
||||
"PHP_SAPI", |
||||
"PHP_OS", |
||||
"PHP_OS_FAMILY", |
||||
"PHP_ZTS", |
||||
"PHP_MAXPATHLEN", |
||||
"PHP_EOL", |
||||
"PATH_SEPARATOR", |
||||
"DIRECTORY_SEPARATOR", |
||||
"PHP_BINDIR", |
||||
"DEFAULT_INCLUDE_PATH", |
||||
"PHP_LIBDIR", |
||||
"PHP_VERSION", |
||||
"PHP_VERSION_ID", |
||||
'PEAR_INSTALL_DIR', |
||||
'PEAR_EXTENSION_DIR', |
||||
'PHP_EXTENSION_DIR', |
||||
'PHP_PREFIX', |
||||
'PHP_MANDIR', |
||||
'PHP_DATADIR', |
||||
'PHP_SYSCONFDIR', |
||||
'PHP_LOCALSTATEDIR', |
||||
'PHP_CONFIG_FILE_PATH', |
||||
'PHP_CONFIG_FILE_SCAN_DIR', |
||||
'PHP_BINARY', |
||||
'LIBXML_VERSION', |
||||
'LIBXML_DOTTED_VERSION', |
||||
'LIBXML_LOADED_VERSION', |
||||
'OPENSSL_VERSION_TEXT', |
||||
'OPENSSL_VERSION_NUMBER', |
||||
'OPENSSL_DEFAULT_STREAM_CIPHERS', |
||||
'PCRE_VERSION', |
||||
'PCRE_VERSION_MAJOR', |
||||
'PCRE_VERSION_MINOR', |
||||
'PCRE_JIT_SUPPORT', |
||||
'STDIN', |
||||
'STDOUT', |
||||
'STDERR', |
||||
]; |
||||
@ -1,9 +1,9 @@ |
||||
#!/usr/bin/env php |
||||
<?php |
||||
require __DIR__ . '/bootstrap.php'; |
||||
require __DIR__ . '/../src/polyfills.php'; |
||||
require __DIR__ . '/../src/gen_stub.php'; |
||||
require __DIR__ . '/../src/compiler.php'; |
||||
require TYPEPHP_ROOT_PATH . '/src/polyfills.php'; |
||||
require TYPEPHP_ROOT_PATH . '/src/gen_stub.php'; |
||||
require TYPEPHP_ROOT_PATH . '/src/compiler.php'; |
||||
|
||||
const TYPEPHP_PHP_SCRIPT_ENTRY = true; |
||||
main($argc, $argv); |
||||
|
||||
@ -1,34 +0,0 @@ |
||||
#!/bin/bash |
||||
# |
||||
# 删除根目录下编译临时产生的 ELF 可执行文件 |
||||
# |
||||
|
||||
DRY_RUN=false |
||||
|
||||
if [ "$1" = "--dry-run" ] || [ "$1" = "-n" ]; then |
||||
DRY_RUN=true |
||||
echo "==> DRY RUN MODE (不会实际删除) <==" |
||||
fi |
||||
|
||||
count=0 |
||||
deleted=0 |
||||
|
||||
while IFS=: read -r path type; do |
||||
case "$type" in |
||||
*ELF*executable*) |
||||
count=$((count + 1)) |
||||
if $DRY_RUN; then |
||||
echo " [DRY RUN] 将删除: $path" |
||||
else |
||||
rm -f "$path" && deleted=$((deleted + 1)) |
||||
echo " 已删除: $path" |
||||
fi |
||||
;; |
||||
esac |
||||
done < <(find "$(dirname "$0")" -maxdepth 1 -type f -exec file {} \; 2>/dev/null) |
||||
|
||||
if $DRY_RUN; then |
||||
echo "==> 共发现 $count 个 ELF 可执行文件(未实际删除)。运行 ./clean-elf.sh 执行删除。" |
||||
else |
||||
echo "==> 共删除 $deleted 个 ELF 可执行文件。" |
||||
fi |
||||
@ -0,0 +1,151 @@ |
||||
# Bash completion for the TypePHP compiler. |
||||
# Generated by: tpc --generate-completion=bash |
||||
|
||||
_typephp_tpc_complete_files() |
||||
{ |
||||
local candidate |
||||
COMPREPLY=() |
||||
while IFS= read -r candidate; do |
||||
if [[ -d "$candidate" || "$candidate" == *.php || "$candidate" == *.yml || "$candidate" == *.yaml || "$candidate" == *.prof ]]; then |
||||
COMPREPLY+=("$candidate") |
||||
fi |
||||
done < <(compgen -f -- "$1") |
||||
} |
||||
|
||||
_typephp_tpc_complete_python_files() |
||||
{ |
||||
local candidate |
||||
COMPREPLY=() |
||||
while IFS= read -r candidate; do |
||||
if [[ -d "$candidate" || "$candidate" == *.py ]]; then |
||||
COMPREPLY+=("$candidate") |
||||
fi |
||||
done < <(compgen -f -- "$1") |
||||
} |
||||
|
||||
_typephp_tpc_complete_paths() |
||||
{ |
||||
local mode="$1" candidate |
||||
COMPREPLY=() |
||||
shift |
||||
while IFS= read -r candidate; do |
||||
COMPREPLY+=("$candidate") |
||||
done < <(compgen "$mode" -- "$1") |
||||
} |
||||
|
||||
_typephp_tpc() |
||||
{ |
||||
local current previous value candidate |
||||
current="${COMP_WORDS[COMP_CWORD]}" |
||||
previous="" |
||||
if (( COMP_CWORD > 0 )); then |
||||
previous="${COMP_WORDS[COMP_CWORD - 1]}" |
||||
fi |
||||
|
||||
local index |
||||
for ((index = 1; index < COMP_CWORD; index++)); do |
||||
if [[ "${COMP_WORDS[index]}" == -- ]]; then |
||||
compopt -o default |
||||
return |
||||
fi |
||||
done |
||||
|
||||
case "$previous" in |
||||
-O) |
||||
COMPREPLY=( $(compgen -W '0 1 2 3' -- "$current") ) |
||||
return |
||||
;; |
||||
--optimize) |
||||
COMPREPLY=( $(compgen -W '0 1 2 3' -- "$current") ) |
||||
return |
||||
;; |
||||
-m) |
||||
COMPREPLY=( $(compgen -W 'bin lib ext' -- "$current") ) |
||||
return |
||||
;; |
||||
--mode) |
||||
COMPREPLY=( $(compgen -W 'bin lib ext' -- "$current") ) |
||||
return |
||||
;; |
||||
--php-version) |
||||
COMPREPLY=( $(compgen -W '8.4 8.5' -- "$current") ) |
||||
return |
||||
;; |
||||
--cxx-std) |
||||
COMPREPLY=( $(compgen -W 'c++17 c++20 c++23' -- "$current") ) |
||||
return |
||||
;; |
||||
--sanitize) |
||||
COMPREPLY=( $(compgen -W 'address undefined' -- "$current") ) |
||||
return |
||||
;; |
||||
--build-dir|--output-dir|-I|--include-path|-L|--link-path) |
||||
compopt -o filenames |
||||
_typephp_tpc_complete_paths -d "$current" |
||||
return |
||||
;; |
||||
--convert-python-to-php) |
||||
compopt -o filenames |
||||
_typephp_tpc_complete_python_files "$current" |
||||
return |
||||
;; |
||||
-o|--output) |
||||
compopt -o filenames |
||||
_typephp_tpc_complete_paths -f "$current" |
||||
return |
||||
;; |
||||
esac |
||||
|
||||
case "$current" in |
||||
-O[0-3]) |
||||
COMPREPLY=("$current") |
||||
return |
||||
;; |
||||
-O*) |
||||
value="${current#-O}" |
||||
COMPREPLY=( $(compgen -W '0 1 2 3' -P '-O' -- "$value") ) |
||||
return |
||||
;; |
||||
--wasm=*) |
||||
value="${current#--wasm=}" |
||||
COMPREPLY=( $(compgen -W 'component browser' -P '--wasm=' -- "$value") ) |
||||
return |
||||
;; |
||||
--generate-completion=*) |
||||
value="${current#--generate-completion=}" |
||||
COMPREPLY=( $(compgen -W 'bash' -P '--generate-completion=' -- "$value") ) |
||||
return |
||||
;; |
||||
--build-dir=*) |
||||
value="${current#--build-dir=}" |
||||
compopt -o filenames |
||||
COMPREPLY=() |
||||
while IFS= read -r candidate; do |
||||
COMPREPLY+=("--build-dir=${candidate}") |
||||
done < <(compgen -d -- "$value") |
||||
return |
||||
;; |
||||
--output-dir=*) |
||||
value="${current#--output-dir=}" |
||||
compopt -o filenames |
||||
COMPREPLY=() |
||||
while IFS= read -r candidate; do |
||||
COMPREPLY+=("--output-dir=${candidate}") |
||||
done < <(compgen -d -- "$value") |
||||
return |
||||
;; |
||||
-* ) |
||||
COMPREPLY=( $(compgen -W '-O --optimize -o --output -h --help -v --version --profile --no-literal-strings --php-version -f --force -m --mode -r --run --debug -j --job --no-console --sanitize --cxx-std --march --target-platform --no-color --build-dir --dry -I --include-path -D --define --no-progress --lto --format -l --link-lib -L --link-path --wasm --wasm= --gen-python-helper --convert-python-to-php --output-dir --output-dir= --build-dir= --generate-completion=' -- "$current") ) |
||||
return |
||||
;; |
||||
esac |
||||
|
||||
compopt -o filenames |
||||
_typephp_tpc_complete_files "$current" |
||||
} |
||||
|
||||
complete -F _typephp_tpc tpc |
||||
complete -F _typephp_tpc ./tpc |
||||
complete -F _typephp_tpc tpc.php |
||||
complete -F _typephp_tpc bin/tpc.php |
||||
complete -F _typephp_tpc ./bin/tpc.php |
||||
File diff suppressed because it is too large
Load Diff
@ -1,81 +0,0 @@ |
||||
# AOT 与 PHP 不兼容特性清单 |
||||
|
||||
本文档只记录当前 AOT 编译器与标准 PHP 不兼容或受限的关键特性。 |
||||
|
||||
## 程序结构 |
||||
|
||||
- 全局作用域不允许可执行语句;只允许声明、`use`、`declare`、常量定义等静态结构。 |
||||
- 函数和方法内部不允许声明函数。 |
||||
- 函数和方法内部不允许声明具名类。 |
||||
- 二进制模式必须定义全局 `main()`。 |
||||
- `main()` 只允许无参数,或 `(int $argc, array $argv)`。 |
||||
- `main()` 必须返回 `void`。 |
||||
|
||||
## 声明与类型 |
||||
|
||||
- 不支持可变变量 `$$var`。 |
||||
- PHP 8.4 property hooks 会降级为 AOT getter/setter;直接属性读写和动态对象读写均受支持。当前不支持对 hook 属性取引用。 |
||||
- 支持 `private(set)` 与 `protected(set)` 非对称属性可见性;在 PHP 8.2~8.4 后端通过自定义属性写 handler 执行同等作用域检查。 |
||||
- 不支持闭包或箭头函数按引用返回。 |
||||
- `__construct()` 不允许返回值。 |
||||
- 参数默认值不允许出现在必填参数之前(`PHP`允许,但会直接丢弃此默认参数)。 |
||||
- 不支持引用可变参数 `&...$args`。 |
||||
- 联合类型、交叉类型、`nullable` 类型仍以 `mixed/any` 作为 C++ 表示,但静态阶段会利用已知表达式类型提前拒绝确定不兼容的参数、返回值和属性赋值;动态值仍保留运行时 type check。 |
||||
- 局部变量类型一旦被静态推断为具体 native 类型,不支持在同一作用域内重新赋值为不兼容类型。 |
||||
- attribute 参数不支持非空数组值和 `new` 表达式。 |
||||
|
||||
## declare |
||||
|
||||
- 不支持 `declare(ticks=...)`。 |
||||
- `declare(encoding=...)` 只允许 `UTF-8`。 |
||||
- `declare(strict_types=...)` 只允许 `strict_types=1`。 |
||||
- 不支持其他 `declare` 指令。 |
||||
|
||||
## 调用与引用 |
||||
|
||||
- TypePHP 使用严格参数数量规则:非 variadic 函数不接受声明范围之外的额外参数;`func_get_args()` 不会隐式放宽签名。 |
||||
- 已知签名的普通函数、普通方法和 native 直调支持引用参数及写回;不要把编译器内部跨 Trait 动态分派的限制误写成“TypePHP 不支持引用参数”。 |
||||
- 闭包和箭头函数不支持引用参数。 |
||||
- 引用赋值不支持从复杂静态属性表达式建立引用。 |
||||
- 动态调用、闭包调用等编译期无法确定参数签名的调用,不能自动转换引用参数;需要显式使用 `refval()` 或等价关键词方法 `toRef()`。 |
||||
- `refval()` / `toRef()` 只接受变量、数组元素或对象属性。 |
||||
- 带 unpack 且尾部追加 named arguments 的调用会退化为动态调用,不能使用 native call。 |
||||
|
||||
## 对象模型 |
||||
|
||||
- `toInt()`、`toString()`、`toArray()` 等保留关键词方法先于普通对象方法解析;需要参数的同名业务方法不按普通对象方法语义调用。 |
||||
- 固定值类型属性未显式初始化时使用类型零值,不保留 ZendPHP 的完整 uninitialized 状态;因此 `??` 等依赖 uninitialized 状态的表达式可能不同。 |
||||
- 禁止子类用同名 `private` 属性隐藏父类私有属性;`public` / `protected` 同名声明视为同一个继承 property slot,仍须满足类型、可见性和 `readonly` 兼容性要求。 |
||||
- 为避免 typed property 写入路径引入额外动态检查,native typed property 在右值类型不确定或与属性类型不一致时会退化为 `setProperty()`;部分标量赋值可能遵循 Zend 弱类型转换,而不是 AOT 默认 strict 语义。 |
||||
- constructor property promotion 的运行时属性可用,但 `ReflectionProperty::isPromoted()` 目前不返回标准 PHP 结果。 |
||||
|
||||
## 表达式与控制流 |
||||
|
||||
- `match` 的 arm condition 不能是 `match` 表达式。 |
||||
- `foreach` by reference 的 value 只能是变量。 |
||||
- `foreach` by reference 不支持 list destructuring。 |
||||
- `std::vector`、`std::map`、`std::ordered_map` 在 `foreach` 期间禁止追加、插入、`unset()` 或整体替换;已有元素的非结构性更新仍可使用赋值运算符完成。 |
||||
- 固定 native typed object property 不允许按 PHP 未初始化语义自由 `unset()`。 |
||||
- native 类型变量执行 `unset()` 不会产生标准 PHP 的变量删除语义。 |
||||
|
||||
## 运行时动态能力 |
||||
|
||||
- `ClassName::class` 只支持字符串字面量或可静态解析的类名。 |
||||
- `static::class` 在需要编译期常量类名的位置不支持。 |
||||
- `__CLASS__` 只允许在 `class` 定义的代码段中使用(`PHP`允许,返回空字符串)。 |
||||
- `__TRAIT__` 只允许在 `trait` 定义的代码段中使用(`PHP`允许,返回空字符串)。 |
||||
- 动态属性链、动态类名、动态函数名和动态回调会统一走 Zend runtime fallback,不保证 native 优化;动态调用的引用参数仍需显式使用 `refval()` 或 `toRef()`。 |
||||
- `Closure::bind()` 绑定静态闭包访问私有成员时,当前行为与标准 PHP 不完全一致。 |
||||
- first-class callable 存入 typed nullable `Closure` 属性后,当前存在运行时稳定性限制。 |
||||
- 所有源文件必须是 `UTF-8` 编码。 |
||||
|
||||
## 编译器自举与内部重构约束 |
||||
|
||||
本节描述编译器自身使用 TypePHP 编译时的约束,不是面向用户代码新增的 PHP 语义差异。 |
||||
|
||||
- 重构前,同一核心类内可静态解析的 `$this->method()` 会生成 native C++ 直调。引用参数会直接映射为 `php::Ref` 或 C++ 引用,写回语义正常。 |
||||
- 将调用方和被调用方拆到不同 Trait 后,单独编译 Trait 本体时无法从 Trait 的 `$this` 确定最终宿主类。当前方法解析器可能将跨 Trait 调用降级为 Zend method call,例如生成 `this_.call(..., php::ArgList{value})`。 |
||||
- 动态 method call 的 `ArgList` 不会仅凭被调 wrapper 的 arginfo 自动把普通实参升级为引用。若被调方法声明 `&$value`,wrapper 会通过 `getCallArgByRef()` 取参;调用方传入的却是普通值,结果是 `must be passed by reference` 警告,并且被调方修改无法写回调用方。 |
||||
- 因此,编译器内部跨 Trait API 禁止使用引用输出参数和“修改传入标量/数组后由调用方读取”的协议。应返回结果值、元组数组或 DTO,例如用 `[$type, $class] = resolveTypeDecl(...)` 代替 `parseTypeDecl(..., &$class)`。 |
||||
- 对字符串累加、数组排序、解析结果输出等内部 helper,优先设计为纯返回值:`$code .= format(...)`、`$files = sort(...)`。只有确认调用会保持 native 直调时,才允许依赖引用写回。 |
||||
- 每次移动方法到 Trait、父类或独立组件后,必须使用自举产物重新编译至少一个覆盖该调用的测试;仅使用 `bin/tpc.php` 运行测试不能发现“源编译器正常、自举编译器退化”的问题。 |
||||
@ -0,0 +1,341 @@ |
||||
# AOT Compilation Speed Optimization Research Notes |
||||
|
||||
This document records the current assessment of Swoole-Compiler AOT compilation speed, bottleneck analysis, and future research directions. It is not yet tied to any specific PR. |
||||
|
||||
## Goal |
||||
|
||||
Reduce the following two categories of time cost: |
||||
|
||||
1. **Cold-start full build**: `./bin/tpc.php project.yml` |
||||
2. **Hot-start incremental build**: recompiling after changing only a few PHP files |
||||
|
||||
The focus is on large projects and compiler self-hosting scenarios. |
||||
|
||||
## Current Pipeline |
||||
|
||||
The main flow is in `src/Php/Translator.php`: |
||||
|
||||
1. `prepare()`: scan files, parse AST, collect symbols, sort dependencies |
||||
2. `convert()`: generate the corresponding `.cc` for each PHP file |
||||
3. `genStubFile()`: generate the arginfo / class register header file |
||||
4. `genFunctionDeclarations()` / `genDataDeclarations()`: generate the build-time internal declaration header |
||||
5. `genExtension()`: generate a single `extension-<target>.cc` |
||||
6. `compile()`: compile all `.cc/.c/...` into `.o` |
||||
7. `build()`: link into the final executable or extension |
||||
|
||||
## Main Bottleneck Assessment |
||||
|
||||
### 1. Common Header Churn Causing Full Recompilation |
||||
|
||||
Currently all translation units include: |
||||
|
||||
- `php_<target>_func_decl.h` |
||||
- `php_<target>_data_decl.h` |
||||
|
||||
As soon as any function declaration, default-argument helper, or global symbol declaration changes, a large number of `.cc` files get recompiled. |
||||
|
||||
This is one of the key reasons for the poor incremental build performance of large projects. |
||||
|
||||
### 2. The `extension-<target>.cc` Single File Is Too Large |
||||
|
||||
The extension main file carries: |
||||
|
||||
- class entry registration |
||||
- the function table |
||||
- literal strings |
||||
- module initialization |
||||
- static property initialization |
||||
- constant initialization |
||||
|
||||
The larger the project, the larger this single TU becomes, making it easy to become a compilation tail bottleneck; even if other files can be compiled in parallel, they all stall on this one large file. |
||||
|
||||
### 3. Missing a General Incremental Cache |
||||
|
||||
Currently only `phpx/src/misc` has an object cache: |
||||
|
||||
- `hasMiscObjectFileCache()` |
||||
|
||||
The `.cc`, common headers, arginfo headers, and extension file generated for the user's own project are still essentially fully regenerated and fully recompiled. |
||||
|
||||
### 4. clang-format Overhead Is Fixed and Serial |
||||
|
||||
`formatCppCode()` runs once for each generated file: |
||||
|
||||
```bash |
||||
clang-format -i <file> |
||||
``` |
||||
|
||||
This introduces: |
||||
|
||||
- extra process startup overhead |
||||
- a large amount of disk I/O |
||||
- serial formatting waits |
||||
|
||||
It is especially noticeable for large projects. |
||||
|
||||
### 5. arginfo / stub Are Regenerated Every Time |
||||
|
||||
`generateStubFile()` currently runs every time; even if the input PHP files have not changed, it regenerates the header files, further amplifying the header churn problem. |
||||
|
||||
### 6. Only the Compilation Stage Is Parallel; the Front Stages Are Mostly Serial |
||||
|
||||
Currently `compileWithPcntl()` only parallelizes `.cc -> .o`: |
||||
|
||||
- prepare |
||||
- convert |
||||
- stub generation |
||||
- format |
||||
|
||||
These stages are still mostly serial. |
||||
|
||||
## Highest-priority Optimization Directions |
||||
|
||||
## P0: Do Not Rewrite Files When Content Is Unchanged |
||||
|
||||
This is the most worthwhile foundational change to prioritize. |
||||
|
||||
### Idea |
||||
|
||||
For all generated files: |
||||
|
||||
- `.cc` |
||||
- arginfo `.h` |
||||
- `php_<target>_func_decl.h` |
||||
- `php_<target>_data_decl.h` |
||||
- `extension-<target>.cc` |
||||
|
||||
Compare the content before writing to disk: |
||||
|
||||
- Same content: **do not write the file** |
||||
- Different content: write it |
||||
|
||||
### Value |
||||
|
||||
Avoid triggering downstream full recompilation merely because of an mtime change. |
||||
|
||||
--- |
||||
|
||||
## P0: General object cache / incremental compilation |
||||
|
||||
Extend the current caching approach that only targets `phpx/src/misc` to user-generated code. |
||||
|
||||
### Suggested Cache Conditions |
||||
|
||||
For each target `.o`: |
||||
|
||||
1. `.o` exists |
||||
2. `.o` is newer than its corresponding source file |
||||
3. `.o` is newer than the headers it depends on |
||||
4. The compile option signature has not changed (optimization level, debug, sanitize, cxxflags, PHP/ZTS, etc.) |
||||
|
||||
When satisfied, skip compilation directly. |
||||
|
||||
### Supporting Requirements |
||||
|
||||
A clear "build signature" mechanism is needed, for example: |
||||
|
||||
- compiler backend |
||||
- cpp compiler path |
||||
- C++ standard |
||||
- optimize/debug/sanitize |
||||
- build mode |
||||
- PHP/ZTS information |
||||
|
||||
--- |
||||
|
||||
## P0: Disable clang-format by Default |
||||
|
||||
It is recommended to make formatting an explicit capability rather than part of the default compilation path. |
||||
|
||||
### Recommendation |
||||
|
||||
- Disable by default |
||||
- Add a `--format` or debug/dev mode to enable it |
||||
- Or format only changed files |
||||
|
||||
### Value |
||||
|
||||
This is a low-risk optimization with immediate effect. |
||||
|
||||
--- |
||||
|
||||
## P1: Split the Common Declaration Header |
||||
|
||||
### Current Problem |
||||
|
||||
Complex default-argument helpers also enter the common `func_decl.h`, widening the impact of header changes. |
||||
|
||||
### Optional Directions |
||||
|
||||
1. **Split declaration headers by source file** |
||||
2. **Move helpers from the common header to local headers / local `.cc`** |
||||
3. **Only truly cross-TU declarations go into the common header** |
||||
|
||||
### Goal |
||||
|
||||
Reduce "one change, full project recompilation". |
||||
|
||||
--- |
||||
|
||||
## P1: Split `extension-<target>.cc` |
||||
|
||||
### Splittable Modules |
||||
|
||||
1. `extension-main.cc` |
||||
2. `extension-class-register-*.cc` |
||||
3. `extension-function-table.cc` |
||||
4. `extension-const-init.cc` |
||||
5. `extension-static-init.cc` |
||||
|
||||
### Value |
||||
|
||||
- Reduce the size of a single TU |
||||
- Enhance parallel compilation benefits |
||||
- Reduce the tail wait for large projects |
||||
|
||||
--- |
||||
|
||||
## P1: arginfo / stub caching |
||||
|
||||
### Direction |
||||
|
||||
Introduce input-content-based caching for `generateStubFile()`: |
||||
|
||||
- Source PHP content hash |
||||
- gen_stub version signature |
||||
- PHP version signature |
||||
|
||||
Do not overwrite output header files when the content is unchanged. |
||||
|
||||
### Value |
||||
|
||||
Reduce header churn, with a clear effect when combined with incremental builds. |
||||
|
||||
--- |
||||
|
||||
## P2: Parallelizing prepare / convert / stub generation |
||||
|
||||
Currently only the compile stage is parallelized. Later the following can be explored: |
||||
|
||||
1. Layering by dependency topology after file scanning |
||||
2. Parallel convert for files in the same layer |
||||
3. Parallel stub generation for files in the same layer |
||||
|
||||
### Risk Points |
||||
|
||||
- There is a lot of shared state (literalStrings, classMap, funcMap, propMap, symbol tables, etc.) |
||||
- It is necessary to first sort out which state can be sharded and which must be merged |
||||
|
||||
Therefore this direction has large benefits but also higher implementation complexity. |
||||
|
||||
--- |
||||
|
||||
## P2: Symbol-dependency-driven minimal recompilation |
||||
|
||||
An ideal incremental build should not be based only on file timestamps, but on: |
||||
|
||||
- which symbols have changed |
||||
- which files depend on those symbols |
||||
|
||||
### Goal |
||||
|
||||
When modifying one PHP file, rebuild only: |
||||
|
||||
1. the file itself |
||||
2. files that depend on its exported symbols |
||||
3. the necessary extension / declaration modules |
||||
|
||||
This would significantly improve hot build speed for large projects. |
||||
|
||||
--- |
||||
|
||||
## P2: Toolchain-level optimization |
||||
|
||||
### Compilation caches |
||||
|
||||
- `ccache` |
||||
- `sccache` |
||||
|
||||
### Faster linkers |
||||
|
||||
- `mold` |
||||
- `lld` |
||||
|
||||
### Precompiled headers |
||||
|
||||
Try PCH for stable large headers, for example: |
||||
|
||||
- `phpx.h` |
||||
- `phpx_helper.h` |
||||
- `phpx_std.h` |
||||
|
||||
These optimizations are relatively cheap to implement and can be advanced together with the compiler option layer. |
||||
|
||||
## Special Constraints on Literal Arrays |
||||
|
||||
Literal arrays are different from literal strings: |
||||
|
||||
- **Literal strings** can leverage permanent strings to bypass the Zend request lifecycle |
||||
- **Literal arrays** must exist from `module_init()` to `module_clean()`, i.e. between PHP's `RINIT/RSHUTDOWN` |
||||
|
||||
Therefore all future "array initialization caching" research must obey: |
||||
|
||||
1. **PHP arrays must not be persisted into process-level permanent objects** |
||||
2. Only the "initialization plan" or "generated code template" can be cached |
||||
3. Real array objects must be constructed within the request lifecycle |
||||
|
||||
The already-introduced `ArrayInitPlan` belongs to this kind of safe abstraction: |
||||
|
||||
- It only saves `expr/init/clean` |
||||
- It does not save array object instances that cross requests |
||||
|
||||
## Suggested Landing Order |
||||
|
||||
### Phase One (Fastest Effect) |
||||
|
||||
1. Do not write files when content is unchanged |
||||
2. General `.o` cache |
||||
3. Disable clang-format by default |
||||
4. arginfo/stub content cache |
||||
|
||||
### Phase Two (Structural Benefits) |
||||
|
||||
5. Split the common header |
||||
6. Split `extension-<target>.cc` |
||||
7. Narrow the visibility of default-argument helpers |
||||
|
||||
### Phase Three (Long-term Optimization) |
||||
|
||||
8. prepare/convert parallelization |
||||
9. Symbol-dependency-driven minimal recompilation |
||||
10. PCH / ccache / mold / sccache |
||||
|
||||
## Suggested Code Locations to Research First |
||||
|
||||
- `src/Php/Translator.php` |
||||
- `formatCppCode()` |
||||
- `compile()` |
||||
- `compileSourceFile()` |
||||
- `compileWithPcntl()` |
||||
- `genFunctionDeclarations()` |
||||
- `genDataDeclarations()` |
||||
- `genExtension()` |
||||
- `genStubFile()` |
||||
- `src/Php/Backend/*` |
||||
- Compile/link command construction, convenient for integrating `ccache` / `mold` / `lld` |
||||
|
||||
## A Realistic Assessment |
||||
|
||||
For large projects, slow AOT compilation is usually not simply "g++ is slow", but the superposition of the following: |
||||
|
||||
1. Full regeneration |
||||
2. Header churn causing full recompilation |
||||
3. A single oversized extension TU |
||||
4. Per-file formatting |
||||
5. Lack of a real incremental cache |
||||
|
||||
Therefore the most effective direction is not to first tune compilation flags, but to prioritize: |
||||
|
||||
- **incrementality** |
||||
- **splitting** |
||||
- **reducing the common dependency surface** |
||||
@ -0,0 +1,42 @@ |
||||
# `#[ArrayDef]` compile-time array contracts |
||||
|
||||
`#[ArrayDef]` attaches key/value type information to a property declared |
||||
exactly as `array`. It supports both Zend classes and `#[Native]` classes and |
||||
has no runtime metadata or per-read overhead. |
||||
|
||||
```php |
||||
class Index |
||||
{ |
||||
#[ArrayDef(Type::String)] |
||||
public array $names = []; // list<string> |
||||
|
||||
#[ArrayDef(Type::Int, Type::String)] |
||||
public array $labels = []; // map<int, string> |
||||
} |
||||
``` |
||||
|
||||
One argument defines a list value type. Two arguments define a map key type |
||||
and value type. Map keys are restricted to `Type::Int` or `Type::String`. |
||||
`ClassName::class` is therefore valid only as a list element type or as the |
||||
second (value) argument of a map. |
||||
|
||||
For direct writes whose expression types are known, the compiler either emits |
||||
the normal write unchanged or reports a fatal type error. An `any` key/value is |
||||
checked with PHPX exact-type helpers at runtime. No coercive `intval()` or |
||||
string conversion is performed. |
||||
|
||||
List writes support `[]` and non-negative integer indexes up to PHP's current |
||||
append position. Indexed writes uniformly emit `php::safeArrayIndex(index, |
||||
array)`. The helper uses `zend_hash_next_free_element()` and follows the |
||||
initial-index rule of `zend_hash_next_index_insert()`, which remains correct |
||||
when `unset()` has created holes or removed the highest numeric key. An index |
||||
equal to that value behaves like the next `$array[]` append; earlier indexes |
||||
may update or refill an element. Negative indexes and indexes beyond the append |
||||
position fail at runtime. There is no AST special case for |
||||
`property[count(property)]`. Maps do not support `[]` append writes. |
||||
|
||||
The contract intentionally applies only to direct element assignment lowered |
||||
by TypePHP. Reads and in-place operators are unchanged. Values passed through |
||||
dynamic functions, callbacks, Reflection, `eval()`, or other ZendVM escape |
||||
paths are outside the contract and have undefined behavior from ArrayDef's |
||||
perspective. |
||||
@ -0,0 +1,662 @@ |
||||
# In-Place Optimization Plan for High-Precision Types |
||||
|
||||
## 1. Background |
||||
|
||||
TypePHP currently implements `BigInt`, `BigFloat`, and `Decimal` as PHPX `Box` objects stored in Zend resources. High-precision operations use an immutable result interface, for example: |
||||
|
||||
```cpp |
||||
target = php::BigInt::mul(target, rhs); |
||||
``` |
||||
|
||||
Even when the PHP source uses compound assignment: |
||||
|
||||
```php |
||||
$target *= $rhs; |
||||
``` |
||||
|
||||
The compiler still generates code that "creates a new result and reassigns". Taking BigInt as an example, a single multiplication currently typically requires: |
||||
|
||||
1. Create a new `BigInt` Box. |
||||
2. Register a new Zend resource. |
||||
3. Initialize a new `mpz_t`. |
||||
4. Allocate GMP limb storage for the computed result. |
||||
5. Move-assign the new resource to the target variable. |
||||
6. Destruct the old resource, Box, and underlying numeric storage. |
||||
|
||||
In scenarios such as loops, accumulation, factorials, and monetary aggregation, these overheads grow linearly with the number of operations: |
||||
|
||||
```php |
||||
for ($i = 0; $i < $count; $i++) { |
||||
$value = $value * 1000; |
||||
} |
||||
``` |
||||
|
||||
The underlying objects of GMP, MPFR, and mpdecimal are not immutable; all three support in-place operations where the output overlaps the input. The current immutable behavior comes from PHPX's high-precision API, not from a limitation of the underlying math libraries. |
||||
|
||||
This document provides the implementation plan, semantic constraints, phased plan, and acceptance criteria for in-place operations on high-precision types. |
||||
|
||||
## 2. Optimization Goals |
||||
|
||||
### 2.1 Primary Goals |
||||
|
||||
- Reuse existing objects for uniquely-held BigInt, BigFloat, and Decimal Boxes. |
||||
- Reuse GMP limb, MPFR mantissa, and mpdecimal coefficient storage as much as possible. |
||||
- Eliminate the result Box and Zend resource temporary objects in compound assignments. |
||||
- Use `php::Int` or `php::Var` directly for native RHS values such as integers, avoiding the construction of high-precision RHS Boxes. |
||||
- Fuse the safe `$x = $x {op} $rhs` pattern into an in-place operation. |
||||
- Preserve PHP's value semantics, reference semantics, evaluation order, and exception behavior. |
||||
- Automatically fall back to the current immutable implementation in unsafe or unprovably-safe scenarios. |
||||
|
||||
### 2.2 Non-Goals |
||||
|
||||
- The first phase does not optimize complex lvalues such as array elements, dynamic properties, and property hooks. |
||||
- It does not rely on whole-program alias analysis to guarantee correctness. |
||||
- It does not modify GMP, MPFR, or mpdecimal third-party source code. |
||||
- It does not turn all ordinary binary expressions into mutable computations. |
||||
- It does not change the implicit conversion rules between different high-precision types. |
||||
|
||||
## 3. Underlying Library Capabilities |
||||
|
||||
| Type | Underlying object | In-place operation | Memory reuse characteristics | |
||||
|---|---|---|---:| |
||||
| BigInt | GMP `mpz_t` / `mpz_class` | Supported | Reuses limb when capacity is sufficient, grows only when the result grows | |
||||
| BigFloat | MPFR `mpfr_t` | Supported | Currently fixed 256-bit precision, ordinary operations can usually keep reusing the mantissa | |
||||
| Decimal | mpdecimal `mpd_t` / `decimal::Decimal` | Supported | Can reuse the coefficient; the library itself provides `operator+=` and other in-place interfaces | |
||||
|
||||
Typical in-place calls are as follows: |
||||
|
||||
```cpp |
||||
mpz_mul(dst, dst, rhs); |
||||
mpfr_mul(dst, dst, rhs, MPFR_RNDN); |
||||
mpd_qmul(dst, dst, rhs, context, &status); |
||||
``` |
||||
|
||||
mpdecimal's C++ wrapper already provides: |
||||
|
||||
```cpp |
||||
Decimal::operator+= |
||||
Decimal::operator-= |
||||
Decimal::operator*= |
||||
Decimal::operator/= |
||||
Decimal::operator%= |
||||
``` |
||||
|
||||
Therefore the technical bottleneck lies mainly in PHP Box sharing semantics, exception safety, and the compiler's evaluation order, rather than in the math libraries themselves. |
||||
|
||||
## 4. Language Semantics That Must Be Preserved |
||||
|
||||
### 4.1 Copy-on-write |
||||
|
||||
A high-precision value may be shared by multiple PHP variables through the same Box: |
||||
|
||||
```php |
||||
$a = std::bigInt(10); |
||||
$b = $a; |
||||
$a *= 2; |
||||
``` |
||||
|
||||
The result must be: |
||||
|
||||
```text |
||||
$a = 20 |
||||
$b = 10 |
||||
``` |
||||
|
||||
The shared Box must not be modified directly. PHPX must check the Zend resource reference count before operating: |
||||
|
||||
- Resource uniquely held: modify the original Box directly. |
||||
- Resource shared: copy the Box, bind the target variable to the copy, then modify the copy. |
||||
|
||||
Runtime copy-on-write is the last line of defense for correctness. Compiler static analysis only reduces unnecessary checks and identifies fusable expressions; it cannot replace the runtime check. |
||||
|
||||
### 4.2 PHP References |
||||
|
||||
In the following scenario, two variables point to the same PHP reference container: |
||||
|
||||
```php |
||||
$a = std::bigInt(10); |
||||
$b =& $a; |
||||
$a *= 2; |
||||
``` |
||||
|
||||
The result must be that both `$a` and `$b` become 20. The in-place API must operate on the actual zval inside the reference through `Variant::unwrap_ptr()`; when copy-on-write occurs, it should update the value in the reference container rather than rebinding the PHPX wrapper object. |
||||
|
||||
### 4.3 RHS and Target Variable Aliasing |
||||
|
||||
The following must be handled correctly: |
||||
|
||||
```php |
||||
$a *= $a; |
||||
``` |
||||
|
||||
It is recommended that the in-place interface take the target by reference and the RHS by value: |
||||
|
||||
```cpp |
||||
BigInt::mulAssign(Variant &target, Variant rhs); |
||||
``` |
||||
|
||||
If the RHS shares the same resource as the target, the RHS's temporary reference count will cause copy-on-write to take the copy branch. This may miss an in-place opportunity, but it naturally guarantees correctness. A dedicated path for "RHS and target are the same Box" can be added later. |
||||
|
||||
### 4.4 Evaluation Order |
||||
|
||||
The following two pieces of code cannot be treated as equivalent in all cases: |
||||
|
||||
```php |
||||
$x = $x * changeValue($x); |
||||
$x *= changeValue($x); |
||||
``` |
||||
|
||||
The RHS may reassign, modify by reference, or modify `$x` through closure capture. C++ function argument evaluation order cannot be used to replace PHP's evaluation rules either. |
||||
|
||||
The compiler must follow these rules: |
||||
|
||||
- Use the existing ordered-operand and side-effect capture mechanism for true `AssignOp`. |
||||
- For `$x = $x {op} $rhs`, fuse only when the RHS does not write to or escape `$x`. |
||||
- Use the current "compute new result then assign" path when safety cannot be proven. |
||||
- If the old `$x` must be saved to preserve ordering, that temporary increases the reference count, and runtime copy-on-write fallback should be allowed automatically. |
||||
|
||||
### 4.5 Complex Lvalues |
||||
|
||||
The following expressions must not be rewritten in the first phase: |
||||
|
||||
```php |
||||
$array[getIndex()] = $array[getIndex()] * 2; |
||||
$object->value = $object->value * 2; |
||||
$object->hooked = $object->hooked * 2; |
||||
``` |
||||
|
||||
Reasons include: |
||||
|
||||
- The subscript expression may execute twice. |
||||
- The number of calls to getters, setters, or property hooks may change. |
||||
- Dynamic property reads/writes may trigger magic methods. |
||||
- The lvalue itself may have side effects. |
||||
|
||||
The first phase only supports simple local variables. Complex lvalues are designed separately in later phases through a "single-evaluation writable target" abstraction. |
||||
|
||||
### 4.6 Exception Safety |
||||
|
||||
The current immutable implementation computes the new result first and only assigns after success, so the target variable remains unchanged when an exception occurs: |
||||
|
||||
```php |
||||
$value = std::decimal('10'); |
||||
|
||||
try { |
||||
$value /= 0; |
||||
} catch (DivisionByZeroError $e) { |
||||
} |
||||
|
||||
echo $value; // still 10 |
||||
``` |
||||
|
||||
The in-place implementation must preserve this behavior. |
||||
|
||||
- BigInt: Check error conditions such as the divisor, modulus, and exponent before modifying. |
||||
- BigFloat: Check division-by-zero and error conditions explicitly defined by the current API before modifying. |
||||
- Decimal: `context.raise(status)` may throw after the underlying result has already been written, requiring a transactional commit or rollback mechanism. |
||||
- Memory allocation failure must also not leave the target in a partially-modified state. |
||||
|
||||
### 4.7 Resource identity |
||||
|
||||
High-precision Boxes are currently exposed as resources, and `get_resource_id()` and strict comparison may observe resource identity. In-place operations keep the resource id for uniquely-held variables, whereas the current immutable implementation generates a new resource id. |
||||
|
||||
One of the following contracts must be clarified before implementation: |
||||
|
||||
1. High-precision types are value types; resource identity is an internal implementation detail and is not guaranteed to remain unchanged across operations. |
||||
2. The current resource identity change must be preserved, in which case only the underlying numeric storage can be reused and the resource must be rewrapped, reducing the benefit. |
||||
|
||||
Option 1 is recommended, and the high-precision type documentation should make it explicit: users should compare values and should not rely on internal resource ids. The value semantics of shared variables are still strictly guaranteed by copy-on-write. |
||||
|
||||
## 5. PHPX Design |
||||
|
||||
### 5.1 Explicit In-Place API |
||||
|
||||
It is not recommended to add high-precision operator overloading to the generic `Variant`. Explicit interfaces should be added to each high-precision type: |
||||
|
||||
```cpp |
||||
class BigInt { |
||||
public: |
||||
static Variant &addAssign(Variant &target, Variant rhs); |
||||
static Variant &subAssign(Variant &target, Variant rhs); |
||||
static Variant &mulAssign(Variant &target, Variant rhs); |
||||
static Variant &divAssign(Variant &target, Variant rhs); |
||||
static Variant &modAssign(Variant &target, Variant rhs); |
||||
}; |
||||
``` |
||||
|
||||
BigFloat and Decimal use the same naming convention. BigInt should also cover bitwise operations and shifts: |
||||
|
||||
```cpp |
||||
bitAndAssign |
||||
bitOrAssign |
||||
bitXorAssign |
||||
bitShiftLeftAssign |
||||
bitShiftRightAssign |
||||
``` |
||||
|
||||
The interface returns `Variant &`, so that compound assignment can still be used as an expression: |
||||
|
||||
```php |
||||
$result = ($value *= 2); |
||||
``` |
||||
|
||||
If the actual generated code is inconvenient to handle the reference return, a statement-only `void` fast path can be provided at the same time, but the assignment expression semantics must not be sacrificed. |
||||
|
||||
### 5.2 Box Uniqueness Utility |
||||
|
||||
Provide a reusable C++17 helper inside PHPX instead of duplicating Zend resource logic across the three types: |
||||
|
||||
```cpp |
||||
template <typename T> |
||||
T *separateBoxForWrite(Variant &target); |
||||
``` |
||||
|
||||
Responsibilities include: |
||||
|
||||
1. Dereference indirect/reference zvals. |
||||
2. Verify that target is the target Box type. |
||||
3. Check the Zend resource reference count. |
||||
4. Return the original Box when uniquely held. |
||||
5. Copy the Box when shared, and update the target through `Variant` assignment semantics. |
||||
6. Preserve typed reference checks and exception propagation. |
||||
|
||||
All three Boxes must support correct copying: |
||||
|
||||
- BigInt: copy the `mpz_class`. |
||||
- BigFloat: initialize at the source precision and copy the `mpfr_t`. |
||||
- Decimal: copy the `decimal::Decimal`. |
||||
|
||||
### 5.3 RHS Extraction |
||||
|
||||
The in-place interface should accept `Variant rhs` directly and reuse the existing operand extractor: |
||||
|
||||
- `php::Int` is converted directly to an underlying integer operand. |
||||
- `php::Var` checks its actual type at runtime. |
||||
- When already a Box of the same type, read the underlying value directly. |
||||
- Strings, floats, and different high-precision types continue to follow the current conversion restrictions. |
||||
|
||||
The generated code should prioritize: |
||||
|
||||
```cpp |
||||
php::BigInt::mulAssign(value, 1000L); |
||||
php::Decimal::mulAssign(value, factor); |
||||
``` |
||||
|
||||
Avoid: |
||||
|
||||
```cpp |
||||
php::BigInt::mulAssign(value, php::toBigInt(1000L)); |
||||
php::Decimal::mulAssign(value, php::toDecimal(1000L)); |
||||
``` |
||||
|
||||
For Decimal's integer RHS, mpdecimal's `_i64`/`_u64` interfaces can be used further to avoid constructing a temporary `decimal::Decimal`: |
||||
|
||||
```cpp |
||||
mpd_qmul_i64(result, left, rhs, context, &status); |
||||
``` |
||||
|
||||
### 5.4 BigInt Implementation Strategy |
||||
|
||||
BigInt prioritizes true in-place operations: |
||||
|
||||
```cpp |
||||
Variant &BigInt::mulAssign(Variant &target, Variant rhs) { |
||||
BigIntOperand right; |
||||
// Extract and validate the RHS first. |
||||
// Then perform copy-on-write on target. |
||||
// Finally call mpz_mul(dst, dst, right). |
||||
return target; |
||||
} |
||||
``` |
||||
|
||||
All recoverable error checks, such as division by zero, modulo by zero, and illegal shift amounts, must be completed before modifying. GMP capacity growth is managed internally; the original limb storage is reused when capacity is sufficient. |
||||
|
||||
### 5.5 BigFloat Implementation Strategy |
||||
|
||||
BigFloat currently uniformly uses `BIG_FLOAT_DEFAULT_PRECISION`, which is suitable for direct in-place operations: |
||||
|
||||
```cpp |
||||
mpfr_mul(dst, dst, rhs, MPFR_RNDN); |
||||
``` |
||||
|
||||
If per-object precision is supported in the future, the relationship between the non-in-place result precision and the compound-assignment target precision must be specified, and tests for objects of different precisions must be added. |
||||
|
||||
### 5.6 Decimal Implementation Strategy |
||||
|
||||
Decimal is implemented in two steps. |
||||
|
||||
The first step uses exception-safe transactional commit: |
||||
|
||||
```cpp |
||||
decimal::Decimal temporary; |
||||
uint32_t status = 0; |
||||
mpd_qmul(temporary.get(), current.getconst(), rhs, context, &status); |
||||
context.raise(status); |
||||
current = std::move(temporary); |
||||
``` |
||||
|
||||
This approach can eliminate the result Box and Zend resource, but still creates an underlying Decimal temporary object. |
||||
|
||||
The second step evaluates true in-place operations: |
||||
|
||||
- Complete explicit checks such as division-by-zero before modifying. |
||||
- Identify which status/trap values may throw after the operation. |
||||
- Provide backup/rollback for operations that may throw, or only perform in-place when it can be proven that no trap will be triggered. |
||||
- Run dedicated tests for Overflow, InvalidOperation, DivisionByZero, and simulated allocation failure. |
||||
|
||||
"Target value partially modified after an exception" must not be accepted for the sake of performance. |
||||
|
||||
## 6. Compiler Design |
||||
|
||||
### 6.1 True Compound Assignment |
||||
|
||||
First modify the existing Big* `AssignOp` generation path: |
||||
|
||||
```php |
||||
$value *= $rhs; |
||||
``` |
||||
|
||||
From: |
||||
|
||||
```cpp |
||||
value = php::BigInt::mul(value, rhs); |
||||
``` |
||||
|
||||
To: |
||||
|
||||
```cpp |
||||
php::BigInt::mulAssign(value, rhs); |
||||
``` |
||||
|
||||
Support matrix: |
||||
|
||||
| Type | First-phase operators | |
||||
|---|---| |
||||
| BigInt | `+= -= *= /= %= &= |= ^= <<= >>=` | |
||||
| BigFloat | `+= -= *= /=` | |
||||
| Decimal | `+= -= *= /= %=` | |
||||
|
||||
### 6.2 Ordinary Assignment Fusion |
||||
|
||||
Identify the following AST: |
||||
|
||||
```php |
||||
$x = $x {op} $rhs; |
||||
``` |
||||
|
||||
Fuse only when all of the following conditions are met: |
||||
|
||||
- The lvalue is a simple named variable. |
||||
- The left operand of the binary expression is the same variable. |
||||
- The variable's static type is BigInt, BigFloat, or Decimal. |
||||
- The operator is in the corresponding type's supported list. |
||||
- The RHS does not contain an assignment to, a reference acquisition of, or a known by-reference argument passing of the target variable. |
||||
- The RHS does not contain `eval`, dynamic calls, or other escape paths that cannot be safely analyzed; or the existing side-effect analysis clearly proves safety. |
||||
- The current expression context can correctly receive the in-place interface's return value. |
||||
|
||||
The following scenarios are not fused in the first phase: |
||||
|
||||
```php |
||||
$x = 2 - $x; |
||||
$x = $x * ($x = 2); |
||||
$x = $x * dynamicCall(); |
||||
$array[$key] = $array[$key] * 2; |
||||
$object->value = $object->value * 2; |
||||
``` |
||||
|
||||
Optimization of commutative operations such as `$x = $rhs + $x` or `$x = $rhs * $x` is deferred to later phases to avoid expanding the scope of the first version. |
||||
|
||||
### 6.3 Failure Fallback |
||||
|
||||
The optimization must be an optional codegen path: |
||||
|
||||
```text |
||||
Can safely operate in-place -> emit *Assign() |
||||
Cannot prove safety -> emit the current new-result path |
||||
``` |
||||
|
||||
Any type uncertainty, complex lvalue, reference escape, or side-effect analysis failure must not cause a compilation error; it should only lose that optimization. |
||||
|
||||
### 6.4 Relationship with SSA/Optimizer |
||||
|
||||
The initial version can perform local AST matching in `AssignOpTrait` and ordinary assignment resolution without relying on a complete SSA. |
||||
|
||||
Later, SSA can provide: |
||||
|
||||
- Whether the target variable has aliases. |
||||
- Whether the RHS writes to the target variable. |
||||
- Whether the variable escapes to dynamic calls or references. |
||||
- Whether it can statically prove the Box is uniquely held. |
||||
|
||||
Even if SSA proves uniqueness, the PHPX runtime copy-on-write check is still recommended to be retained, unless there is a strict escape proof and dedicated tests. |
||||
|
||||
## 7. Phased Implementation Plan |
||||
|
||||
### Phase 0: Baseline and Observation |
||||
|
||||
- Add test helper facilities for counting high-precision Box/resource creation. |
||||
- Establish benchmarks for BigInt, BigFloat, and Decimal loop operations. |
||||
- Record current wall time, Box count, resource count, and underlying allocation count. |
||||
- Freeze the current aliasing, reference, exception, and resource identity behavior. |
||||
|
||||
Deliverable: a baseline report and behavior tests, with no change to generated code. |
||||
|
||||
### Phase 1: Native RHS Fast Path |
||||
|
||||
- BigInt operations directly accept `php::Int`. |
||||
- BigFloat operations directly accept `php::Int`, `php::Float`. |
||||
- Decimal operations directly accept `php::Int` and `php::Var` that is actually an int. |
||||
- The Decimal integer path prioritizes `mpd_q*_i64`. |
||||
- Eliminate the high-precision Box the compiler creates for the RHS. |
||||
|
||||
Deliverable: no more unnecessary `toBigInt()`, `toBigFloat()`, `toDecimal()` on the RHS. |
||||
|
||||
### Phase 2: PHPX Copy-on-write Infrastructure |
||||
|
||||
- Implement `separateBoxForWrite<T>()`. |
||||
- Complete copy tests for the three Box types. |
||||
- Cover ordinary variables, shared variables, PHP references, indirect zvals, and RHS being the same Box. |
||||
- Clarify the resource identity contract. |
||||
|
||||
Deliverable: standalone PHPX unit tests, with no modification to the compiler generation path. |
||||
|
||||
### Phase 3: BigInt and BigFloat Compound Assignment |
||||
|
||||
- Implement the BigInt `*Assign()` method family. |
||||
- Implement the BigFloat `*Assign()` method family. |
||||
- Modify the generated code for true PHP `AssignOp`. |
||||
- Preserve fallback for unsafe paths. |
||||
- Run the full PHPX test suite, full compiler PHPUnit, relevant PHPT, and bootstrap compilation. |
||||
|
||||
Deliverable: syntax such as `$x *= $rhs` uses true in-place operations. |
||||
|
||||
### Phase 4: Ordinary Assignment Fusion |
||||
|
||||
- Identify simple local variables `$x = $x {op} $rhs`. |
||||
- Implement target variable write/escape checks. |
||||
- Prioritize enabling for pure-literal and pure-variable RHS. |
||||
- Preserve the old path for RHS with side effects. |
||||
|
||||
Deliverable: common patterns in the problem description no longer require users to manually convert to compound assignment. |
||||
|
||||
### Phase 5: Decimal Transactional In-Place Interface |
||||
|
||||
- Implement the Decimal `*Assign()` API. |
||||
- First use "underlying temporary result + commit on success". |
||||
- Optimize integer RHS using the `_i64` fast path. |
||||
- Cover all Decimal traps and the target value after exceptions. |
||||
|
||||
Deliverable: eliminate the Decimal result Box/resource while maintaining strong exception safety. |
||||
|
||||
### Phase 6: Decimal True In-Place Computation |
||||
|
||||
- Analyze the status/trap values each operator may trigger. |
||||
- Directly use the target `mpd_t` for operations that can be proven safe. |
||||
- Preserve the transactional path for high-risk operations. |
||||
- Determine through benchmarks whether the complexity is worthwhile. |
||||
|
||||
Deliverable: common Decimal accumulation operations reuse coefficient storage. |
||||
|
||||
### Phase 7: Complex Lvalues and Further Optimizations |
||||
|
||||
- Design a single-evaluation writable target abstraction. |
||||
- Evaluate support for array elements, static properties, and ordinary properties. |
||||
- Property hooks, magic methods, and dynamic properties are not enabled by default unless the number of calls and ordering can be strictly preserved. |
||||
- Evaluate commutative expression fusion and SSA uniqueness proof. |
||||
|
||||
## 8. Test Plan |
||||
|
||||
### 8.1 PHPX Unit Tests |
||||
|
||||
Each type and each operator must at least cover: |
||||
|
||||
- Unique Box in-place update. |
||||
- Shared Box triggers copy-on-write. |
||||
- PHP references update the same referenced value. |
||||
- RHS and target are the same Box. |
||||
- Allowed RHS types such as Int, Float, String, and Var. |
||||
- Exceptions for illegal RHS types. |
||||
- Edge cases such as division by zero, modulo by zero, and negative exponents. |
||||
- The target value remains unchanged after an exception. |
||||
- Capacity growth triggered by extremely large numbers. |
||||
- Multiple consecutive operations. |
||||
|
||||
### 8.2 Compiler PHPUnit |
||||
|
||||
Check the generated code: |
||||
|
||||
- `AssignOp` generates calls such as `BigInt::mulAssign()`. |
||||
- `$x = $x * 1000` is fused. |
||||
- RHS native integers no longer construct Big* Boxes. |
||||
- No fusion when the RHS has side effects. |
||||
- Array elements and properties are not fused in the first phase. |
||||
- Unsupported operators continue to produce the original FatalError. |
||||
|
||||
### 8.3 PHPT |
||||
|
||||
At least cover: |
||||
|
||||
```php |
||||
$a *= 2; |
||||
$a = $a * 2; |
||||
$b = $a; $a *= 2; |
||||
$b =& $a; $a *= 2; |
||||
$a *= $a; |
||||
$a *= ($factor = 2); |
||||
$result = ($a *= 2); |
||||
``` |
||||
|
||||
And cover for the three high-precision types: |
||||
|
||||
- Positive, negative, and zero values. |
||||
- Extreme values and precision boundaries. |
||||
- All supported compound assignment operators. |
||||
- The lvalue after an exception. |
||||
- Consecutive updates in a loop. |
||||
|
||||
### 8.4 Integration Verification |
||||
|
||||
Each phase must at least execute: |
||||
|
||||
```bash |
||||
./vendor/bin/phpunit |
||||
php run-tests.php tests/compiler/bigint tests/compiler/bignumber tests/compiler/decimal |
||||
php bin/tpc.php project.yml |
||||
``` |
||||
|
||||
PHPX modifications must also run the full PHPX unit test suite. |
||||
|
||||
## 9. Performance Acceptance |
||||
|
||||
Performance tests must at least include: |
||||
|
||||
- Sizes of 1, 4, 16, 64, 256, and 1024 limb/decimal digits. |
||||
- RHS being small integers, same-type high-precision values, and dynamic `php::Var`. |
||||
- Unique Box and shared Box. |
||||
- Loops of 1 thousand, 100 thousand, and 1 million iterations. |
||||
- BigInt growth multiplication versus stable-capacity addition. |
||||
- BigFloat fixed-precision accumulation. |
||||
- Decimal fixed 50-digit precision accumulation. |
||||
|
||||
Functional acceptance criteria: |
||||
|
||||
- Compound assignment of a unique BigInt/BigFloat does not create a result Box/resource per iteration. |
||||
- Native RHS does not create a high-precision Box. |
||||
- Shared Box correctly triggers copy-on-write. |
||||
- All exception paths keep the target value unchanged. |
||||
- Bootstrap compilation and full test suites pass. |
||||
|
||||
Performance acceptance is based on baseline data and does not preset unrealistic fixed multiples. At least the following should be reported separately: |
||||
|
||||
- Total elapsed time. |
||||
- Box/resource creation counts. |
||||
- Underlying memory allocation counts and bytes. |
||||
- Peak memory. |
||||
- Copy-on-write hit rate and fallback rate. |
||||
|
||||
If an optimization path cannot reduce allocations, or causes clear regression in common non-in-place expressions, the old path should be retained or that sub-optimization should be reverted. |
||||
|
||||
## 10. Risks and Rollback Strategy |
||||
|
||||
Main risks: |
||||
|
||||
- Incorrect Box sharing determination causing other variables to be modified unexpectedly. |
||||
- References or indirect zvals being rebound instead of updated. |
||||
- RHS side effects changing the evaluation order. |
||||
- Decimal target value being polluted after an exception. |
||||
- Undocumented changes in resource identity behavior. |
||||
- In-place capacity growth failure leaving an invalid underlying object. |
||||
|
||||
Control measures: |
||||
|
||||
- All optimizations are concentrated in a standalone PHPX API and a single compiler codegen branch. |
||||
- Fall back to the old implementation when safety cannot be proven. |
||||
- Enable incrementally by type and by operator. |
||||
- Commit each phase independently, avoiding modifying too many semantics at once. |
||||
- Do not remove the existing immutable API until exception, aliasing, and reference tests are complete. |
||||
|
||||
Rollback only requires the compiler to regenerate: |
||||
|
||||
```cpp |
||||
target = Type::operation(target, rhs); |
||||
``` |
||||
|
||||
The original immutable API must be retained throughout the entire migration period. |
||||
|
||||
## 11. Recommended Priority |
||||
|
||||
Considering benefit, complexity, and risk, the recommended order is: |
||||
|
||||
1. BigFloat in-place compound assignment. |
||||
2. BigInt in-place compound assignment. |
||||
3. BigInt/BigFloat ordinary assignment fusion. |
||||
4. Decimal native integer RHS fast path. |
||||
5. Decimal transactional `*Assign()`. |
||||
6. Decimal true in-place computation. |
||||
7. Complex lvalues and SSA enhancements. |
||||
|
||||
BigFloat has fixed precision and is the easiest to stably reuse underlying memory; BigInt has broader applications and its overall benefit may be the largest; Decimal has the most complex exception and trap semantics, and its true in-place modification should be pushed last. |
||||
|
||||
## 12. Final Target Code |
||||
|
||||
For safe simple variables: |
||||
|
||||
```php |
||||
$value = $value * 1000; |
||||
``` |
||||
|
||||
The final generation: |
||||
|
||||
```cpp |
||||
php::BigInt::mulAssign(value, 1000L); |
||||
``` |
||||
|
||||
Runtime: |
||||
|
||||
```text |
||||
Unique Box: reuse Box, resource, and underlying storage in place |
||||
Shared Box: copy-on-write, then modify the new Box |
||||
Unsafe scenario: fall back to the current immutable result implementation |
||||
``` |
||||
|
||||
This design confines the performance optimization within verifiable boundaries while preserving the consistency of TypePHP with PHP assignment, reference, and exception semantics. |
||||
@ -0,0 +1,142 @@ |
||||
# TypePHP Compiler Command Line |
||||
|
||||
## Bash Autocompletion |
||||
|
||||
TypePHP provides Bash completion that is kept in sync with the current compiler arguments. To enable it temporarily in the current terminal: |
||||
|
||||
```shell |
||||
source <(./tpc --generate-completion=bash) |
||||
``` |
||||
|
||||
When developing from the source repository, you can also run `source completions/tpc.bash` directly. |
||||
|
||||
To install it for the current user and have it auto-loaded in subsequent Bash sessions: |
||||
|
||||
```shell |
||||
mkdir -p "$HOME/.local/share/bash-completion/completions" |
||||
./tpc --generate-completion=bash \ |
||||
> "$HOME/.local/share/bash-completion/completions/tpc" |
||||
``` |
||||
|
||||
If your system does not automatically scan the user completion directory, you can load it in `~/.bashrc`: |
||||
|
||||
```shell |
||||
source "$HOME/.local/share/bash-completion/completions/tpc" |
||||
``` |
||||
|
||||
For a system-wide installation, write the generated output to `/usr/share/bash-completion/completions/tpc`. This operation typically |
||||
requires root privileges. |
||||
|
||||
The completion supports build options, WASM profiles, build modes, PHP/C++ versions, sanitizers, input sources, |
||||
project YAML, Python source files, and directory arguments. Everything after `--` is treated as arguments of the compiled program itself, and the completer |
||||
does not interpret them as `tpc` arguments anymore. |
||||
|
||||
Release packages ship a pre-generated `completions/tpc.bash`. This file is produced by the same generator, with unit |
||||
tests ensuring it matches the output of `./tpc --generate-completion=bash`. |
||||
|
||||
This document is kept in sync with `src/Translator.php::showUsage()`. Usage: |
||||
|
||||
```bash |
||||
bin/tpc.php <file|dir|project.yml> [options] [-- program-args...] |
||||
``` |
||||
|
||||
## Common Examples |
||||
|
||||
```bash |
||||
# Compile a single file |
||||
bin/tpc.php app.php |
||||
|
||||
# Optimize and run; arguments after `--` are passed to the generated program |
||||
bin/tpc.php app.php -O2 -r -- --flag value |
||||
|
||||
# Compile a project configuration |
||||
bin/tpc.php project.yml -O2 -j 8 |
||||
|
||||
# Generate a PHP extension |
||||
bin/tpc.php extension/ -m ext -o my_extension |
||||
|
||||
# Only generate C++, without compiling and linking |
||||
bin/tpc.php app.php --dry --build-dir /tmp/typephp-build |
||||
``` |
||||
|
||||
## Build Options |
||||
|
||||
| Option | Description | |
||||
|---|---| |
||||
| `-O <0-3>` | Optimization level, default `0`. | |
||||
| `-d`, `--debug` | Debug build; disables optimization and adds debug symbols and TypePHP source tracking. | |
||||
| `-o`, `--output <file>` | Output file name. | |
||||
| `-m`, `--mode <bin|lib|ext>` | Build mode, default `bin`. | |
||||
| `-r`, `--run` | Run after a successful build. | |
||||
| `-j`, `--job <num>` | Number of parallel compilation jobs, default `4`. | |
||||
| `-f`, `--force` | Ignore the phpx misc object cache and force recompilation. | |
||||
| `--build-dir <dir>` | Directory for generated C++ and intermediate artifacts. | |
||||
| `--dry` | Only generate C++, skipping compilation and linking. | |
||||
| `--format` | Run clang-format on the generated code. | |
||||
| `--no-progress` | Do not show the progress bar; output progress per file. | |
||||
| `--no-color` | Disable colored output. | |
||||
|
||||
`-v` / `--version` only displays the version; it is not a verbose option. |
||||
|
||||
## Target and Toolchain |
||||
|
||||
| Option | Description | |
||||
|---|---| |
||||
| `--php-version <8.4|8.5>` | Restrict the accepted PHP syntax version, default `8.5`. | |
||||
| `--cxx-std <ver>` | C++ standard, e.g. `c++17`, `c++20`. | |
||||
| `--march <arch>` | Target instruction set, e.g. `native`, `x86-64-v3`. | |
||||
| `--target-platform <triple>` | Cross-compilation target triple. | |
||||
| `--lto` | Enable Link Time Optimization. | |
||||
| `--sanitize <type>` | Enable a sanitizer, e.g. `address`, `undefined`. | |
||||
| `--no-console` | Windows GUI mode hides the console window. | |
||||
| `--profile` | Enable the gperftools profiler on Linux and force recompilation of related objects. | |
||||
|
||||
`--php-version` controls the source syntax accepted by the parser and is also used in `project.yml` to select source files based on `PHP_VERSION` / `PHP_VERSION_ID`. It is not responsible for choosing the PHP installation directory to link against. |
||||
|
||||
The minimum runtime version for both TypePHP and PHPX is PHP 8.4. `--php-version` and the actually linked `libphp.so` do not need to match exactly in minor version, but both must be PHP 8.4 or higher. |
||||
|
||||
## C++ Compilation and Link Arguments |
||||
|
||||
These arguments can all be repeated: |
||||
|
||||
```bash |
||||
-I /opt/library/include |
||||
-D FEATURE_ENABLED=1 |
||||
-L /opt/library/lib |
||||
-l curl |
||||
``` |
||||
|
||||
Corresponding long options: |
||||
|
||||
- `--include-path` |
||||
- `--define` |
||||
- `--link-path` |
||||
- `--link-lib` |
||||
|
||||
## Project Configuration Precedence |
||||
|
||||
When a `project.yml` is passed, command-line arguments take precedence over same-named settings in the YAML. For the project file format, see the user documentation and the project configuration parser in the code. |
||||
|
||||
### PHP Extension Dependencies |
||||
|
||||
When a program depends on other PHP extensions, the required modules can be written into the Zend module dependency table: |
||||
|
||||
```yaml |
||||
extension-dependencies: |
||||
- pdo_mysql |
||||
- curl |
||||
``` |
||||
|
||||
`ext-deps` is an equivalent shorthand name. Only one of these names can be used in a project; using both `extension-dependencies` and `ext-deps` produces a configuration error. |
||||
|
||||
The compiler generates a `ZEND_MOD_REQUIRED` for each entry. Zend checks whether these extensions are loaded when loading the TypePHP module. This setting does not represent native link libraries; C/C++ link dependencies still use `link-libs`. |
||||
|
||||
## Viewing the Authoritative Help |
||||
|
||||
The command-line implementation may continue to evolve; for released versions the actual arguments are determined by the following command: |
||||
|
||||
```bash |
||||
bin/tpc.php --help |
||||
``` |
||||
|
||||
For compatibility boundaries, see [INCOMPATIBLE_PHP_FEATURES.md](INCOMPATIBLE_PHP_FEATURES.md); for build modes, see [COMPILATION_MODES.md](COMPILATION_MODES.md). |
||||
@ -0,0 +1,96 @@ |
||||
# AOT compile-time functions and keyword methods |
||||
|
||||
This document records the compile-time functions, keyword methods, and related construction entry points that are specific to the AOT compiler. They are not part of standard PHP syntax, and an ordinary PHP runtime can only rely on the compatibility stubs provided by `src/polyfills.php`. |
||||
|
||||
## Core compile-time functions |
||||
|
||||
There are currently 5 core global compile-time functions. |
||||
|
||||
| Name | Parameters | Purpose | Current primary handling location | |
||||
| --- | --- | --- | --- | |
||||
| `any($value)` | 1 | Degrades the expression to `mixed/any`, preventing further processing as a static native/object type. | General function-call expression entry. | |
||||
| `refval($target)` | 1 | Explicitly passes a variable, array element, or object property by reference to a dynamic call or a call whose reference parameter cannot be statically identified. | Argument parsing, dynamic calls, SSA/optimizer reference escape analysis. | |
||||
| `objval($value, ClassName::class or 'ClassName')` | 2 | Tells the compiler that `$value` is an object of the specified class, and generates the `php::toObject(..., target_ce)` runtime fallback check. | Function-call resolution, object type inference. | |
||||
| `expected($condition)` | 1 | Marks the condition as usually true, generating the Zend `EXPECTED(...)` branch prediction macro. | General function-call expression entry. | |
||||
| `unexpected($condition)` | 1 | Marks the condition as usually false, generating the Zend `UNEXPECTED(...)` branch prediction macro. | General function-call expression entry. | |
||||
|
||||
Constraints: |
||||
|
||||
- `refval()` only accepts variables, array elements, or object properties. |
||||
- The second parameter of `objval()` must be a compile-time-resolvable class-name string or `ClassName::class`. |
||||
- `any()` can be used in any expression position; it directly expands its single argument at compile time without generating a runtime function call. |
||||
- `expected()` / `unexpected()` accept exactly one non-expanded argument and return bool; they are usually used in `if`, `elseif`, and loop conditions, and do not change the argument's evaluation count or true/false semantics. |
||||
|
||||
## Keyword methods |
||||
|
||||
There are currently 12 built-in keyword methods. |
||||
|
||||
| Name | Equivalent behavior | Description | |
||||
| --- | --- | --- | |
||||
| `toAny()` | `any($receiver)` | Returns the receiver itself, but with the type degraded to `mixed/any`. | |
||||
| `toRef()` | `refval($receiver)` | Returns a reference to the receiver; parameter restrictions are the same as `refval()`. | |
||||
| `toObject()` | `php::toObject($receiver)` | May take a target-class parameter, performing object conversion/checking. | |
||||
| `toInt()` | `php::toInt($receiver)` | Converts to a native int expression. | |
||||
| `toFloat()` | `php::toFloat($receiver)` | Converts to a native float expression. | |
||||
| `toString()` | `php::toString($receiver)` | Converts to a string expression. | |
||||
| `toBool()` | `php::toBool($receiver)` | Converts to a bool expression. | |
||||
| `toArray()` | `php::toArray($receiver)` | Converts to an array expression. | |
||||
| `toStream()` | `php::toStream($receiver)` | Converts to a stream expression. | |
||||
| `toBigInt()` | `php::BigInt::newInstance($receiver)` | Constructs a BigInt. | |
||||
| `toBigFloat()` | `php::BigFloat::newInstance($receiver)` | Constructs a BigFloat. | |
||||
| `toDecimal()` | `php::Decimal::newInstance($receiver)` | Constructs a Decimal. | |
||||
|
||||
Constraints: |
||||
|
||||
- `toAny()` and `toRef()` accept no parameters. |
||||
- `toRef()` only applies to receivers that can take references. |
||||
- Keyword methods take precedence over ordinary methods and universal method dispatch. |
||||
|
||||
## `std::` compile-time construction entry points |
||||
|
||||
There are currently 10 `std::` compile-time construction entry points. |
||||
|
||||
| Name | Purpose | Main limitation | |
||||
| --- | --- | --- | |
||||
| `std::int($value)` | Explicitly creates a native int expression. | Requires 1 value parameter. | |
||||
| `std::float($value)` | Explicitly creates a native float expression. | Requires 1 value parameter. | |
||||
| `std::bool($value)` | Explicitly creates a native bool expression. | Requires 1 value parameter. | |
||||
| `std::bigInt($value)` | Constructs a BigInt. | Implicit construction from a float variable is not allowed. | |
||||
| `std::decimal($value)` | Constructs a Decimal. | A float variable must be converted via string or integer; float literals are handled per the original literal. | |
||||
| `std::bigFloat($value)` | Constructs a BigFloat. | Requires 1 value parameter. | |
||||
| `std::array($type, $size[, ...$sizes])` | Constructs a fixed-size std array. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
| `std::vector($type[, $size])` | Constructs a std vector. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
| `std::map($keyType, $valueType)` | Constructs a std map. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
| `std::ordered_map($keyType, $valueType)` | Constructs a std ordered map. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
|
||||
## Std container conversion keyword methods |
||||
|
||||
There are currently 4 Std container conversion keyword methods. |
||||
|
||||
| Name | Purpose | Main limitation | |
||||
| --- | --- | --- | |
||||
| `toStdArray(...)` | Wraps the variable as a std array. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
| `toStdVector(...)` | Wraps the variable as a std vector. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
| `toStdMap(...)` | Wraps the variable as a std map. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
| `toStdOrderedMap(...)` | Wraps the variable as a std ordered map. | Can only be used in the top-level scope of the variable's first assignment. | |
||||
|
||||
## Mechanisms not counted in this list |
||||
|
||||
- `$array->any()` is a universal method that maps to PHP `array_any()`, not the `any()` compile-time function. |
||||
- `Type::*` are compile-time type-description constants, not functions. |
||||
- keyword extension methods are a user-defined extension method mechanism and are not part of the fixed built-in compile-time function list. |
||||
|
||||
## Implementation constraints |
||||
|
||||
Compile-time functions should be usable in any legal expression position and maintain consistent semantics across all paths: |
||||
|
||||
- `any()` is already handled uniformly at the ordinary function-call expression entry; assignments, parameters, return values, array elements, and operator subexpressions share the same semantics. |
||||
- `refval()` / `toRef()` have many special cases in argument parsing and dynamic call paths and should later be unified into a single "reference-wrapping expression" resolution entry. |
||||
- `objval()` is currently recognized through the function-call resolution and type-inference paths and is relatively centralized. |
||||
- `expected()` / `unexpected()` generate `EXPECTED(...)` / `UNEXPECTED(...)` respectively at the ordinary function-call entry and produce no PHP runtime function call. |
||||
|
||||
Future refactoring goals: |
||||
|
||||
- Establish a unified `CompileTimeFunctionResolver` or equivalent module. |
||||
- Reuse the same compile-time function metadata in `parseExpr()` / `detectTypeOfExpr()` / `detectClassOfExpr()` / argument parsing paths. |
||||
- Continue unifying the behavior of `refval()` and `objval()` across different expression paths. |
||||
@ -0,0 +1,474 @@ |
||||
# TypePHP Core Class OOA / OOD / OOP Refactoring Plan |
||||
|
||||
## 1. Document Purpose |
||||
|
||||
This document guides the subsequent architectural refactoring of `Translator`, `CompilerBase`, and `Preprocessor`. Implementation should proceed phase by phase, without a one-shot rewrite. |
||||
|
||||
Current baseline: |
||||
|
||||
| Class | Lines | Methods | Current Role | |
||||
|---|---:|---:|---| |
||||
| `Translator` | 3717 | 126 | CLI, project configuration, code generation, build coordination | |
||||
| `CompilerBase` | 3843 | 208 | Compilation state, AST dispatch, Resolver, Emitter | |
||||
| `Preprocessor` | 973 | 25 | Declaration collection, AST lowering, dependency and semantic validation | |
||||
|
||||
Current inheritance structure: |
||||
|
||||
```text |
||||
Translator |
||||
extends Preprocessor |
||||
extends CompilerBase |
||||
``` |
||||
|
||||
Main problems: |
||||
|
||||
- The three classes form an inheritance-based God Object, where high-level flows can access all low-level mutable state. |
||||
- A large number of Traits only achieve physical splitting, and still implicitly depend on all `$this` fields of the host. |
||||
- Frontend analysis, name resolution, semantic validation, code generation, and native build lack clear boundaries. |
||||
- Arrays, AST attributes, and `string|false` are used as implicit protocols between modules. |
||||
- Manual state switches such as `resetFile()`, `resetClass()`, `resetFunction()` are easy to forget to restore. |
||||
|
||||
## 2. Refactoring Principles |
||||
|
||||
1. Behavior preservation takes priority; separate architecture refactoring from semantic changes into separate commits. |
||||
2. Establish object boundaries first, then remove old entry points; during migration, old and new implementations may coexist. |
||||
3. Prefer composition, interfaces, and immutable value objects; business Traits only as a transitional measure. |
||||
4. Handler Registry indexes directly by node class name, avoiding linear responsibility chains that degrade compilation performance. |
||||
5. Compilation state must be passed explicitly through Context or Session. |
||||
6. Resolver is responsible for decisions; Generator/Emitter is responsible for code generation; the two must not be mixed. |
||||
7. Each phase must have independent PHPUnit and corresponding PHPT regression evidence. |
||||
|
||||
## 3. OOA: Domain Object Analysis |
||||
|
||||
### 3.1 Compilation Session Domain |
||||
|
||||
Responsible for the state of one compilation lifecycle: |
||||
|
||||
```text |
||||
CompilationSession |
||||
CompilerConfiguration |
||||
ScopeStack |
||||
ScopeFrame |
||||
FileContext |
||||
ClassContext |
||||
FunctionContext |
||||
``` |
||||
|
||||
### 3.2 Frontend Analysis Domain |
||||
|
||||
Responsible for PHP source code to validated AST/model: |
||||
|
||||
```text |
||||
SourceParser |
||||
FrontendPipeline |
||||
DeclarationCollector |
||||
DependencyAnalyzer |
||||
SemanticAnalyzer |
||||
AstLoweringPass |
||||
``` |
||||
|
||||
### 3.3 Resolution Domain |
||||
|
||||
Responsible for symbol and language semantic decisions: |
||||
|
||||
```text |
||||
NameResolver |
||||
TypeResolver |
||||
MethodCallResolver |
||||
PropertyResolver |
||||
ConstantResolver |
||||
AccessPolicy |
||||
SymbolRepository |
||||
InheritanceGraph |
||||
``` |
||||
|
||||
### 3.4 Code Generation Domain |
||||
|
||||
Responsible for AST/entity model to C++: |
||||
|
||||
```text |
||||
ExpressionCompiler |
||||
StatementCompiler |
||||
ClassCodeGenerator |
||||
FunctionCodeGenerator |
||||
WrapperGenerator |
||||
ExtensionModuleGenerator |
||||
``` |
||||
|
||||
### 3.5 Build Domain |
||||
|
||||
Responsible for generated files to final artifacts: |
||||
|
||||
```text |
||||
SourcePipeline |
||||
NativeBuilder |
||||
ResourceCompiler |
||||
BuildModeStrategy |
||||
CompileOptions |
||||
LinkOptions |
||||
BuildResult |
||||
``` |
||||
|
||||
### 3.6 Application Entry Domain |
||||
|
||||
Responsible for user input and top-level flow: |
||||
|
||||
```text |
||||
CompilerApplication |
||||
CompileCommand |
||||
CompilerFacade |
||||
ProjectYamlLoader |
||||
CommandLineInput |
||||
``` |
||||
|
||||
## 4. OOD: Target Architecture |
||||
|
||||
```text |
||||
CompilerApplication |
||||
└─ CompilerFacade |
||||
├─ ProjectLoader |
||||
├─ SourcePipeline |
||||
├─ FrontendPipeline |
||||
│ ├─ DeclarationCollector |
||||
│ ├─ AstLoweringPass[] |
||||
│ ├─ DependencyAnalyzer |
||||
│ └─ SemanticAnalyzer |
||||
├─ TranslationCodeGenerator |
||||
│ ├─ ExpressionCompiler |
||||
│ ├─ StatementCompiler |
||||
│ ├─ ClassCodeGenerator |
||||
│ └─ FunctionCodeGenerator |
||||
└─ NativeBuilder |
||||
``` |
||||
|
||||
### 4.1 Translator's Target |
||||
|
||||
`Translator` ultimately acts as a Facade/Coordinator, only organizing the flow: |
||||
|
||||
```php |
||||
final class Translator |
||||
{ |
||||
public function translate(ProjectInput $input): BuildResult; |
||||
} |
||||
``` |
||||
|
||||
It is forbidden to continue assuming: |
||||
|
||||
- CLI argument parsing; |
||||
- AST node semantic determination; |
||||
- C++ template concatenation for classes, functions, and wrappers; |
||||
- shell command execution; |
||||
- preprocessor internal state. |
||||
|
||||
### 4.2 Preprocessor's Target |
||||
|
||||
`Preprocessor` becomes an independent Frontend Service, no longer extending `CompilerBase`: |
||||
|
||||
```php |
||||
final class Preprocessor |
||||
{ |
||||
public function process(SourceUnit $source, CompilationSession $session): PreprocessResult; |
||||
} |
||||
``` |
||||
|
||||
Organize Passes using the Pipeline pattern: |
||||
|
||||
```text |
||||
ParseSourcePass |
||||
→ NameResolutionPass |
||||
→ PropertyHookLoweringPass |
||||
→ DeclarationCollectionPass |
||||
→ TraitExpansionPass |
||||
→ InheritanceValidationPass |
||||
→ TypeValidationPass |
||||
→ DependencyCollectionPass |
||||
``` |
||||
|
||||
### 4.3 CompilerBase's Target |
||||
|
||||
`CompilerBase` is eventually replaced by the following objects: |
||||
|
||||
- `CompilationSession`: compilation lifecycle state; |
||||
- `ExpressionCompiler`: expression Handler dispatch; |
||||
- `StatementCompiler`: statement Handler dispatch; |
||||
- `CompilerServices`: the Resolver and Generator collections; |
||||
- `CodeGenerationContext`: generation-phase context. |
||||
|
||||
After the migration is complete, delete `CompilerBase`, or keep only a short-term compatibility Facade. |
||||
|
||||
## 5. Design Pattern Application |
||||
|
||||
### Facade |
||||
|
||||
`CompilerFacade` and the final `Translator` provide a stable top-level entry point, hiding the details of Frontend, Generator, and Builder. |
||||
|
||||
### Pipeline |
||||
|
||||
`FrontendPipeline` explicitly maintains the order of frontend Passes, and each Pass can be tested independently. |
||||
|
||||
### Handler Registry |
||||
|
||||
Expressions and statements are dispatched in O(1) by AST class name: |
||||
|
||||
```php |
||||
$handlers[Expr\MethodCall::class] = $methodCallHandler; |
||||
``` |
||||
|
||||
### Strategy |
||||
|
||||
Build modes are implemented by the following strategies: |
||||
|
||||
- `BinaryBuildStrategy` |
||||
- `ExtensionBuildStrategy` |
||||
- `LibraryBuildStrategy` |
||||
- `EmbedBuildStrategy` |
||||
|
||||
### Chain of Responsibility |
||||
|
||||
Method resolution order: |
||||
|
||||
```text |
||||
DeclaredMethodResolver |
||||
→ ObjectExtensionMethodResolver |
||||
→ UniversalMethodResolver |
||||
→ MagicCallResolver |
||||
→ DynamicCallResolver |
||||
``` |
||||
|
||||
Property resolution order: |
||||
|
||||
```text |
||||
BackingSlotResolver |
||||
→ PropertyHookResolver |
||||
→ DeclaredPropertyResolver |
||||
→ NativePropertyResolver |
||||
→ DynamicPropertyResolver |
||||
``` |
||||
|
||||
### Repository |
||||
|
||||
`SymbolRepository` uniformly manages functions, classes, interfaces, constants, and inheritance relationships; callers no longer handle Repository keys themselves. |
||||
|
||||
### State / Scope Stack |
||||
|
||||
Use `ScopeStack` and `ScopeGuard` to replace the reset series of methods, ensuring state restoration on exceptions, `Skip`, and `Redo`. |
||||
|
||||
### Value Object / Result Object |
||||
|
||||
Gradually introduce: |
||||
|
||||
- `SourceUnit` |
||||
- `SourceLocation` |
||||
- `GeneratedExpression` |
||||
- `GeneratedStatement` |
||||
- `ResolvedCall` |
||||
- `ResolvedPropertyAccess` |
||||
- `PreprocessResult` |
||||
- `TranslationResult` |
||||
- `BuildResult` |
||||
|
||||
## 6. OOP Incremental Implementation Phases |
||||
|
||||
### Phase 0: Architecture Protection Tests |
||||
|
||||
Tasks: |
||||
|
||||
- Establish an expression and statement node coverage checklist; |
||||
- Add frontend Pass order tests; |
||||
- Add Scope exception recovery tests; |
||||
- Add SymbolRepository name normalization tests; |
||||
- Fix the Property Hook, extension methods, inheritance, and exception test sets; |
||||
- Establish snapshots or structural assertions for key generated C++. |
||||
|
||||
Acceptance: |
||||
|
||||
- PHPUnit passes in full; |
||||
- Core PHPT all pass; |
||||
- Subsequent phases can identify Handler omissions and evaluation order changes. |
||||
|
||||
### Phase 1: CompilationSession and ScopeStack |
||||
|
||||
Tasks: |
||||
|
||||
1. Create `CompilationSession`, `CompilerConfiguration`, `ScopeStack`. |
||||
2. Move in current file, namespace, class, method, function, PHP version, and phase state. |
||||
3. `CompilerBase`'s old properties first proxy to the Session. |
||||
4. Replace the reset series of methods with `enter/leave` and `try/finally`. |
||||
5. Delete the proxy properties. |
||||
|
||||
Acceptance: |
||||
|
||||
- `CompilerBase` no longer directly owns scope state; |
||||
- Exceptions, `Skip`, `Redo` do not pollute the next scope; |
||||
- ScopeStack has independent unit tests. |
||||
|
||||
### Phase 2: Preprocessor Pipeline |
||||
|
||||
Tasks: |
||||
|
||||
1. Create `FrontendPass` and `FrontendPipeline`. |
||||
2. First migrate Property Hook lowering. |
||||
3. Migrate declaration collection and namespace/use handling. |
||||
4. Migrate dependency collection and file ordering. |
||||
5. Migrate Trait, inheritance, override, and interface implementation validation. |
||||
6. Remove `Preprocessor extends CompilerBase`. |
||||
|
||||
Acceptance: |
||||
|
||||
- Each Pass has independent tests; |
||||
- Pass order is defined in only one place; |
||||
- `Preprocessor.php` is kept within 200–300 lines. |
||||
|
||||
### Phase 3: ExpressionCompiler |
||||
|
||||
Tasks: |
||||
|
||||
1. Create `ExpressionHandlerRegistry` and `GeneratedExpression`. |
||||
2. Migrate scalar/const/variable, unary/binary/cast, array/assign in order. |
||||
3. Migrate function/method/static calls. |
||||
4. Migrate property, nullsafe, isset/empty/ref. |
||||
5. Migrate closure, generator, fiber, new, clone, instanceof. |
||||
6. Delete the old large `parseExpr()` dispatch. |
||||
|
||||
Acceptance: |
||||
|
||||
- Every supported Expr has a unique Handler; |
||||
- Handlers do not depend on `CompilerBase`; |
||||
- Registry checks for duplicates and omissions at startup; |
||||
- Evaluation order and side effect tests all pass. |
||||
|
||||
### Phase 4: StatementCompiler |
||||
|
||||
Tasks: |
||||
|
||||
1. Create `StatementHandlerRegistry` and `GeneratedStatement`. |
||||
2. Migrate return/echo, conditionals, loops, exception control flow. |
||||
3. Migrate global/static/namespace/declare. |
||||
4. Eliminate the shared `beforeStmtLines`, `afterStmtLines` protocol. |
||||
|
||||
Acceptance: |
||||
|
||||
- Statement Handlers return an explicit Result; |
||||
- Control flow generation is removed from `CompilerBase`; |
||||
- before/after statements are composed through Result. |
||||
|
||||
### Phase 5: Resolver Chain |
||||
|
||||
Tasks: |
||||
|
||||
1. Establish `MethodCallResolverChain`. |
||||
2. Establish `PropertyResolverChain`. |
||||
3. Establish `ConstantResolverChain`. |
||||
4. Establish a unified `AccessPolicy`. |
||||
5. Migrate `MethodCallTrait`, `PropertyAccessTrait`, `UniversalMethodCall`, `MagicMethodDetector` into the Resolver. |
||||
|
||||
Acceptance: |
||||
|
||||
- The priority of normal methods, extension methods, and `__call()` is defined in only one place; |
||||
- The priority of backing slot, Property Hook, and normal properties is defined in only one place; |
||||
- `private(set)`, `protected(set)` are determined only by AccessPolicy; |
||||
- Resolver no longer returns `string|false`. |
||||
|
||||
### Phase 6: Independent Code Generators |
||||
|
||||
Tasks: |
||||
|
||||
- Establish `ClassCodeGenerator`; |
||||
- Establish `FunctionCodeGenerator`; |
||||
- Establish `WrapperGenerator`; |
||||
- Establish `ExtensionModuleGenerator`; |
||||
- Move `parseClass()`, `parseFunction()`, wrapper, and registration code out of `Translator`. |
||||
|
||||
Acceptance: |
||||
|
||||
- Generator input is Entity/IR, output is `GeneratedFile`; |
||||
- `Translator` no longer directly concatenates concrete C++ templates; |
||||
- Key generation results have snapshot tests. |
||||
|
||||
### Phase 7: Translator Facade |
||||
|
||||
Tasks: |
||||
|
||||
1. Move CLI to `CompilerApplication` / `CompileCommand`. |
||||
2. `Translator` only injects ProjectLoader, Frontend, CodeGenerator, NativeBuilder. |
||||
3. Remove `Translator extends Preprocessor`. |
||||
4. Converge the public entry point to `translate(ProjectInput): BuildResult`. |
||||
|
||||
Acceptance: |
||||
|
||||
- `Translator` does not parse CLI; |
||||
- does not directly access the AST; |
||||
- does not directly execute shell; |
||||
- does not depend on Preprocessor internal state; |
||||
- the file is kept within 300–500 lines. |
||||
|
||||
### Phase 8: Remove the CompilerBase Inheritance Hierarchy |
||||
|
||||
Tasks: |
||||
|
||||
1. Clear remaining compatibility proxies and business Traits. |
||||
2. Move public query interfaces into explicit Services/Contexts. |
||||
3. Delete the `Translator → Preprocessor → CompilerBase` inheritance chain. |
||||
4. Delete uncalled legacy methods and fields. |
||||
|
||||
Acceptance: |
||||
|
||||
- Core components collaborate only through interfaces and DTOs; |
||||
- No business Trait implicitly accesses all host state; |
||||
- No core class exceeds about 800 lines; |
||||
- PHPUnit, core PHPT, and multi-PHP-version builds all pass. |
||||
|
||||
## 7. Per-Phase Execution Template |
||||
|
||||
Each phase is implemented following these steps: |
||||
|
||||
1. List the methods, fields, and call sites to migrate. |
||||
2. Add protection tests first. |
||||
3. Create the new interfaces, DTOs, and implementation. |
||||
4. Change old entry points to delegate to the new implementation. |
||||
5. Migrate call sites in batches. |
||||
6. Delete old implementations and proxy fields. |
||||
7. Run syntax checks, PHPUnit, and corresponding PHPT. |
||||
8. Check `git diff --check` and untracked build artifacts. |
||||
9. Update the phase status and actual deviations in this document. |
||||
|
||||
## 8. Recommended Directory Layout |
||||
|
||||
```text |
||||
src/ |
||||
├─ Application/ |
||||
├─ Compiler/ |
||||
│ └─ Scope/ |
||||
├─ Frontend/ |
||||
│ ├─ Pass/ |
||||
│ └─ Result/ |
||||
├─ CodeGeneration/ |
||||
│ ├─ Expression/ |
||||
│ └─ Statement/ |
||||
├─ Resolver/ |
||||
│ ├─ Call/ |
||||
│ ├─ Property/ |
||||
│ ├─ Constant/ |
||||
│ └─ Access/ |
||||
├─ Symbol/ |
||||
├─ Build/ |
||||
│ ├─ BuildMode/ |
||||
│ └─ Options/ |
||||
└─ Diagnostics/ |
||||
``` |
||||
|
||||
## 9. Phase Status |
||||
|
||||
| Phase | Status | |
||||
|---|---| |
||||
| 0. Architecture protection tests | Not started | |
||||
| 1. CompilationSession / ScopeStack | Not started | |
||||
| 2. Preprocessor Pipeline | Not started | |
||||
| 3. ExpressionCompiler | Not started | |
||||
| 4. StatementCompiler | Not started | |
||||
| 5. Resolver Chain | Not started | |
||||
| 6. Independent code generators | Not started | |
||||
| 7. Translator Facade | Not started | |
||||
| 8. Remove CompilerBase inheritance hierarchy | Not started | |
||||
|
||||
This table should be continuously updated during implementation; a phase must not be declared complete based solely on file line count. |
||||
@ -0,0 +1,356 @@ |
||||
# C++ Namespace, Prefix, and Symbol ABI Rules |
||||
|
||||
This document is the internal C++ naming convention for TypePHP, PHPX, and TypePHP-generated code. It solves the following problems: |
||||
|
||||
- Distinguishing TypePHP runtime logic, PHPX ZendAPI wrappers, project-private implementations, and user PHP symbols; |
||||
- Preventing framework helpers from generating the same C++ symbols as user-defined PHP functions or class methods; |
||||
- Clarifying which names are part of the stable ABI and which names are only for internal use within a single generated project; |
||||
- Providing a unified naming decision for adding helpers, caches, entry functions, and generated symbols. |
||||
|
||||
## 1. Overall Rules |
||||
|
||||
| Naming Domain | Meaning | Typical Form | Visibility Scope | ABI Property | |
||||
| --- | --- | --- | --- | --- | |
||||
| `typephp_` | TypePHP-specific runtime or compiled-artifact support logic | `typephp_call_parent_constructor()` | TypePHP/PHPX runtime | Internal or explicitly exported ABI | |
||||
| `php::` | C++ wrappers for PHP runtime capabilities such as ZendAPI, zval, HashTable, and call frames | `php::deindirect()` | PHPX C++ API | PHPX API | |
||||
| `typephp_<project>` | The private C++ namespace of a single compiled project | `namespace typephp_tpc` | Current generated project | Non-public ABI | |
||||
| `php_` | C++ callable symbols mapped from user PHP functions and class methods | `php_app__user__save()` | Visible to the linker | TypePHP/stub callable ABI | |
||||
|
||||
Core constraints: |
||||
|
||||
1. Do not add new global framework `php_*` helpers. |
||||
2. Capabilities that are unrelated to TypePHP and only wrap ZendAPI must be placed in `namespace php`. |
||||
3. Logic that is unique to TypePHP and needs to be called across generated files uses the `typephp_` prefix. |
||||
4. Data and functions that serve only one compiled project go into the `typephp_<project>` namespace. |
||||
5. Global `php_*` callable names are reserved for the compiled ABI of user PHP declarations. |
||||
|
||||
## 2. `typephp_`: TypePHP-specific Logic |
||||
|
||||
`typephp_` indicates that the API's semantics are defined by TypePHP and are not a general-purpose C++ wrapper of ZendAPI. Common scenarios include: |
||||
|
||||
- TypePHP property read/write rules; |
||||
- TypePHP construction, cloning, and parent method call chains; |
||||
- Runtime support for TypePHP compile-time Attributes; |
||||
- TypePHP-specific runtime logic such as Native Class and Property Hook; |
||||
- Initialization and shutdown entry points of the TypePHP embed runtime. |
||||
|
||||
Examples: |
||||
|
||||
```cpp |
||||
typephp_call_parent_constructor(object, constructor, args); |
||||
typephp_call_parent_clone(object, clone_method); |
||||
typephp_install_property_handlers(class_entry, handlers); |
||||
typephp_write_property_scoped(object, member, value, scope); |
||||
TYPEPHP_RUNTIME_INIT(project)(argc, argv); |
||||
``` |
||||
|
||||
### 2.1 Usage Boundaries |
||||
|
||||
- This prefix is the TypePHP internal C/C++ name space and does not represent PHP user functions. |
||||
- When adding an API, use a complete, recognizable snake_case name; do not use overly broad names such as `typephp_call()`. |
||||
- Functions used in only one `.cc` file should additionally be marked `static` or placed in an anonymous namespace. |
||||
- When crossing dynamic library boundaries, use the corresponding export macro; helpers that do not need to be exported should not widen symbol visibility. |
||||
- Do not use `typephp_` merely because the code is in `typephp_helper.h`; the criterion is whether the semantics are TypePHP-specific. |
||||
|
||||
### 2.2 Positive and Negative Examples |
||||
|
||||
```cpp |
||||
// Correct: the constructor chain semantics are TypePHP-specific. |
||||
typephp_call_parent_constructor(object, constructor, args); |
||||
|
||||
// Incorrect: this only materializes an INDIRECT zval into a plain value and is not TypePHP-specific. |
||||
typephp_deindirect(value); |
||||
|
||||
// Correct: generic Zend value wrapping belongs to PHPX. |
||||
php::deindirect(value); |
||||
``` |
||||
|
||||
## 3. `php::`: C++ Wrappers for ZendAPI |
||||
|
||||
`namespace php` is provided by PHPX to wrap Zend's C API, macros, raw pointers, and manual resource management into a type-safe, RAII-friendly C++ API. |
||||
|
||||
This naming domain contains two categories of capabilities: |
||||
|
||||
1. PHP values and runtime objects, such as `php::Var`, `php::Str`, `php::Array`, and `php::Object`; |
||||
2. Safe wrappers of ZendAPI, such as symbol lookup, scope management, value conversion, object creation, and invocation. |
||||
|
||||
Examples: |
||||
|
||||
```cpp |
||||
php::Var value; |
||||
php::Array arguments; |
||||
|
||||
auto plain = php::deindirect(value); |
||||
auto called_ce = php::getCalledCe(this_); |
||||
auto scope = php::getCallableScope(function, this_); |
||||
auto create_object = php::getCreateObjectFn(class_entry); |
||||
auto globals = php::globalsArray(); |
||||
``` |
||||
|
||||
### 3.1 When to Use `php::` |
||||
|
||||
Place into `namespace php` when the following conditions are met: |
||||
|
||||
- The API is meaningful to any PHPX C++ caller; |
||||
- The API's behavior can be fully explained by Zend/PHP runtime semantics; |
||||
- The API does not depend on TypePHP AST, compile-time Attributes, or TypePHP-specific language rules; |
||||
- The API's main purpose is to hide Zend macros, raw `zval *`, reference counting, or exception checking. |
||||
|
||||
### 3.2 Forbidding Global `php_*` Helpers |
||||
|
||||
The following legacy forms are forbidden: |
||||
|
||||
```cpp |
||||
php::Var php_deindirect(const php::Var &value); |
||||
php::Str php_get_called_class(php::Object &this_); |
||||
zend_class_entry *php_get_called_ce(php::Object &this_); |
||||
auto php_get_create_object_fn(zend_class_entry *ce); |
||||
``` |
||||
|
||||
They must be written as: |
||||
|
||||
```cpp |
||||
namespace php { |
||||
|
||||
Var deindirect(const Var &value); |
||||
Str getCalledClass(Object &this_); |
||||
zend_class_entry *getCalledCe(Object &this_); |
||||
auto getCreateObjectFn(zend_class_entry *ce); |
||||
|
||||
} // namespace php |
||||
``` |
||||
|
||||
The reason is that users can legitimately declare: |
||||
|
||||
```php |
||||
function deindirect(mixed $value): mixed {} |
||||
function get_called_ce(): string {} |
||||
function get_create_object_fn(): string {} |
||||
``` |
||||
|
||||
These PHP functions generate `php_deindirect`, `php_get_called_ce`, and |
||||
`php_get_create_object_fn`. If PHPX also defines same-named helpers globally, conflicts may occur at the declaration, overload resolution, or linking stage. |
||||
|
||||
### 3.3 Naming Style |
||||
|
||||
The PHPX C++ API uses the existing camelCase style: |
||||
|
||||
```cpp |
||||
php::getCalledClass(); |
||||
php::getClassEntrySafe(); |
||||
php::getPersistentCache(); |
||||
php::stdCreateObject(); |
||||
``` |
||||
|
||||
Do not mechanically preserve Zend's snake_case names as global C++ names. Lower-level calls can continue to use the original Zend API, such as `zend_objects_new()`, but the wrapper layer exposed to generated code should use `php::`. |
||||
|
||||
## 4. `typephp_<project>`: Project-private Namespace |
||||
|
||||
Each TypePHP compiled project has an independent C++ namespace: |
||||
|
||||
```text |
||||
typephp_<target-name> |
||||
``` |
||||
|
||||
For example, if the project name is `tpc`: |
||||
|
||||
```cpp |
||||
namespace typephp_tpc { |
||||
// Project-private generated state and helpers. |
||||
} |
||||
``` |
||||
|
||||
The `-` and `*` in the project name are converted to `_`, and the remaining characters must satisfy the compiler's target identifier validation. Because of the fixed `typephp_` prefix, the final C++ namespace is a valid identifier even if the project name starts with a digit. |
||||
|
||||
### 4.1 Content That Should Go into This Namespace |
||||
|
||||
- The literal string table and `get_str()`; |
||||
- The class/function/property cache tables and their accessor functions; |
||||
- Global variable storage of the current project; |
||||
- Class entries, object handlers, and default property templates; |
||||
- Module entry and MINIT/RINIT/RSHUTDOWN auxiliary state; |
||||
- Functions such as `module_init()` and `module_clean()` that are called only inside the generated extension file; |
||||
- Project-level generated state such as the Python module cache. |
||||
|
||||
Illustration: |
||||
|
||||
```cpp |
||||
namespace typephp_demo { |
||||
|
||||
static php::Str literal_strings[] = { |
||||
php::Str{"hello"}, |
||||
}; |
||||
|
||||
php::Str &get_str(uint32_t index) { |
||||
return literal_strings[index]; |
||||
} |
||||
|
||||
static THREAD_LOCAL zend_class_entry *class_map[8]; |
||||
|
||||
zend_class_entry *get_class(int id, const php::Str &name) { |
||||
// Resolve and cache a symbol owned by this project. |
||||
} |
||||
|
||||
static void module_init() { |
||||
// Initialize this project's generated state. |
||||
} |
||||
|
||||
} // namespace typephp_demo |
||||
``` |
||||
|
||||
### 4.2 Visibility and ABI |
||||
|
||||
- Names inside `typephp_<project>` are implementation details, not library stub ABI. |
||||
- Objects and functions that can be limited to `static` should continue to be marked `static`. |
||||
- Generated headers may declare project-internal accessors that must be used across translation units, but must not expose underlying arrays or cache tables. |
||||
- External handwritten C++ code must not depend on literal indexes, cache indexes, or project-internal storage names. |
||||
- Different TypePHP projects can be linked into the same process, because the same internal short names reside in different project namespaces. |
||||
|
||||
### 4.3 Scope Takes Priority over Name Spelling |
||||
|
||||
Historical generated names may still appear in the project namespace, for example: |
||||
|
||||
```cpp |
||||
typephp_demo::php_class_entry_App_User |
||||
``` |
||||
|
||||
Although the member name starts with `php_`, the full symbol resides in `typephp_demo`, so it is a project-private implementation rather than the global user callable ABI described in Section 5. New project-internal helpers should prefer short names without `php_`, such as `get_class()`, `get_func()`, and `get_str()`. |
||||
|
||||
## 5. `php_`: The C++ ABI of User PHP Callables |
||||
|
||||
The global `php_` prefix is used by TypePHP to map user-declared PHP functions and class methods into C++ callable symbols. This naming is used by generated code, library stubs, and external C++ implementations alike, so it cannot be changed arbitrarily. |
||||
|
||||
Example: |
||||
|
||||
```php |
||||
namespace App; |
||||
|
||||
function greet(string $name): string {} |
||||
|
||||
class User |
||||
{ |
||||
public function save(): bool {} |
||||
} |
||||
``` |
||||
|
||||
The conceptual C++ symbols are: |
||||
|
||||
```cpp |
||||
php::Str php_app__greet(php::Str name); |
||||
php::Bool php_app__user__save(php::Object &this_); |
||||
``` |
||||
|
||||
The rules include: |
||||
|
||||
- Use `php_` to mark "mapped from a PHP declaration"; |
||||
- PHP namespace, class, and method/function names are combined after normalization; |
||||
- `__` is the existing ABI combination separator; |
||||
- The first parameter of an instance method is the object `this_`; |
||||
- Stubs, libraries, and consumers must use exactly the same mapping rules. |
||||
|
||||
### 5.1 Why Internal Helpers Cannot Use `php_` |
||||
|
||||
The `php_` mapping is not an independent reserved keyword space, but a mechanical ABI of user PHP names. The following user declaration: |
||||
|
||||
```php |
||||
function deindirect(mixed $value): mixed {} |
||||
``` |
||||
|
||||
naturally generates: |
||||
|
||||
```cpp |
||||
php::Var php_deindirect(php::Var value); |
||||
``` |
||||
|
||||
Therefore, if the framework defines a global `php_deindirect()`, it encroaches on the user symbol space. The correct approach is `php::deindirect()`. |
||||
|
||||
### 5.2 Combination Collisions |
||||
|
||||
Because the current ABI uses `__` to combine PHP namespace, class, and callable names, the following two PHP declarations may map to the same C++ symbol: |
||||
|
||||
```php |
||||
function App\user__test(): void {} |
||||
|
||||
namespace App; |
||||
class User |
||||
{ |
||||
public function test(): void {} |
||||
} |
||||
``` |
||||
|
||||
The compiler must detect this situation during the preprocessing stage and throw a FatalError; it must not be handled through overriding, link order, or added runtime dispatch. Changing the mapping separator rules would break existing stubs/ABI, so collisions must be resolved by the user through renaming. |
||||
|
||||
### 5.3 Entry Symbol Exceptions |
||||
|
||||
A small number of C ABI/embed entry points are fixed by the generator and do not belong to ordinary user callables. For example: |
||||
|
||||
```cpp |
||||
php_<project>_embed_get_module(); |
||||
typephp_<project>_runtime_init(argc, argv); |
||||
typephp_<project>_runtime_shutdown(); |
||||
``` |
||||
|
||||
These are the connection points between the binary/library embed runtime and the current project's module entry. Definitions and references are uniformly generated through |
||||
`TYPEPHP_EMBED_GET_MODULE_FUNCTION()`, `TYPEPHP_RUNTIME_INIT_FUNCTION()`, |
||||
`TYPEPHP_RUNTIME_SHUTDOWN_FUNCTION()`, and the corresponding symbol macros, in a style consistent with Zend's |
||||
`PHP_MINIT_FUNCTION()`/`PHP_MINIT()`. The final symbols contain the project name and must not be used as a general helper naming template. |
||||
|
||||
### 5.4 The Shared Runtime in Multi-extension Processes |
||||
|
||||
TypePHP extensions must not separately compile or statically link a PHPX implementation containing process-level Zend state. The Reflection handler, |
||||
`FiberGenerator` class entry, scope, and Property Hook runtime are all provided solely by the shared `libphpx`: |
||||
|
||||
- Host-mode extensions/libraries must link `libphpx.so`, `libphpx.dylib`, or `phpx.dll`, and must not fall back to `libphpx.a`; |
||||
- Unix PHP extensions do not link the Embed `libphp.so`; Zend/PHP symbols are provided by the SAPI that loads them; |
||||
- macOS extensions use `-undefined dynamic_lookup` to resolve host symbols; |
||||
- Binaries and standalone WASI programs can still link statically, because each process or Wasm instance has only one copy of the runtime. |
||||
|
||||
`src/core/typephp_*.cc` only carries the TypePHP-specific `typephp_*` runtime; `php::` ZendAPI wrappers should be placed in core source files without the |
||||
`typephp_` prefix, such as `src/core/scope.cc`. |
||||
|
||||
## 6. Name Selection Flow |
||||
|
||||
When adding a C++ API, judge in the following order: |
||||
|
||||
1. **Is it the compiled body of a user PHP function or class method?** |
||||
- Yes: use the existing `php_` callable ABI generator; do not handwrite another mapping. |
||||
2. **Does it serve only one current TypePHP project?** |
||||
- Yes: place it in `typephp_<project>`, and use `static` or private accessors where possible. |
||||
3. **Does it implement TypePHP-specific semantics?** |
||||
- Yes: use the `typephp_` prefix. |
||||
4. **Is it only a C++ wrapper of Zend/PHP runtime capabilities?** |
||||
- Yes: place it in `namespace php`, using the PHPX camelCase style. |
||||
5. **None of the above?** |
||||
- It should not be arbitrarily added to `typephp_helper.h`; reconfirm its owning module and public API boundary. |
||||
|
||||
## 7. Code Review Checklist |
||||
|
||||
When adding or modifying generated helpers, check: |
||||
|
||||
- [ ] No new global `php_*` helpers in `typephp_helper.h`; |
||||
- [ ] ZendAPI wrappers are in `namespace php`; |
||||
- [ ] TypePHP-specific logic uses `typephp_`; |
||||
- [ ] Project caches and storage are in `typephp_<project>`; |
||||
- [ ] Project-private tables are not exposed directly via `extern` through generated headers; |
||||
- [ ] User callables still use the unified `php_` ABI generator; |
||||
- [ ] New names do not collide with user-declarable PHP functions or methods; |
||||
- [ ] bin, lib, ext, and WASM builds use the same project name derivation rule; |
||||
- [ ] Stub and existing ABI are evaluated together when modifying the public callable mapping; |
||||
- [ ] At least one compilation regression test is added for a user function with the same name. |
||||
|
||||
The current related regression test is: |
||||
|
||||
```text |
||||
tests/compiler/basic/helper-symbol-collision.phpt |
||||
``` |
||||
|
||||
## 8. Main Implementation Locations |
||||
|
||||
| Responsibility | File | |
||||
| --- | --- | |
||||
| `php_` callable prefix and combination separator | `src/CompilerBase.php` | |
||||
| Callable combination collision detection | `src/Preprocessor.php` | |
||||
| `typephp_<project>` generation and project-private tables | `src/Translator.php` | |
||||
| TypePHP extension prefix constants | `src/Metadata/Constants.php` | |
||||
| PHPX/TypePHP helper classification | `vendor/swoole/phpx/include/typephp_helper.h` | |
||||
| Embed module accessor concatenation | `vendor/swoole/phpx/src/misc/typephp_main.cc` | |
||||
@ -0,0 +1,99 @@ |
||||
# GMP Function Comparison Table (not implemented) |
||||
|
||||
This document records the functions of the PHP GMP extension that have not yet been implemented in the BigInt type, as a reference for future development. |
||||
|
||||
## Statistics |
||||
|
||||
The GMP extension has 44 functions in total (excluding `gmp_init` and the alias `gmp_div`). 17 are covered, 27 are not covered. |
||||
|
||||
## Implemented |
||||
|
||||
| GMP function | BigInt method / operator | Description | |
||||
|----------|---------------------|------| |
||||
| `gmp_init` | `std::bigInt()` | construction | |
||||
| `gmp_add` | `add()` / `+` | addition | |
||||
| `gmp_sub` | `sub()` / `-` | subtraction | |
||||
| `gmp_mul` | `mul()` / `*` | multiplication | |
||||
| `gmp_div_q` | `div()` / `/` | division (quotient) | |
||||
| `gmp_div_r` | `mod()` / `%` | division (remainder) | |
||||
| `gmp_div_qr` | `divmod()` | quotient and remainder | |
||||
| `gmp_mod` | `mod()` / `%` | modulo | |
||||
| `gmp_pow` | `pow()` | power | |
||||
| `gmp_powm` | `powmod()` | modular exponentiation | |
||||
| `gmp_neg` | `neg()` / `-` (unary) | negation | |
||||
| `gmp_abs` | `abs()` | absolute value | |
||||
| `gmp_sqrt` | `sqrt()` | square root | |
||||
| `gmp_gcd` | `gcd()` | greatest common divisor | |
||||
| `gmp_cmp` | `cmp()` / `<=>` | comparison | |
||||
| `gmp_and` | `bitAnd()` / `&` | bitwise AND | |
||||
| `gmp_or` | `bitOr()` / `\|` | bitwise OR | |
||||
| `gmp_xor` | `bitXor()` / `^` | bitwise XOR | |
||||
| `gmp_com` | `bitNot()` / `~` | bitwise NOT | |
||||
| `gmp_testbit` | `testBit()` | bit test | |
||||
| `gmp_popcount` | `popCount()` | population count | |
||||
| `gmp_intval` | `toInt()` | to int | |
||||
| `gmp_strval` | `toString()` | to string | |
||||
|
||||
## Not implemented (sorted by priority) |
||||
|
||||
### High priority — commonly used number-theory functions |
||||
|
||||
| GMP function | Suggested method name | Signature | Description | |
||||
|----------|-----------|------|------| |
||||
| `gmp_sign` | `sign()` | `(): int` | sign, returns -1/0/1 | |
||||
| `gmp_lcm` | `lcm($x)` | `(BigInt): BigInt` | least common multiple | |
||||
| `gmp_perfect_square` | `perfectSquare()` | `(): bool` | whether it is a perfect square | |
||||
| `gmp_perfect_power` | `perfectPower()` | `(): bool` | whether it is a perfect power | |
||||
| `gmp_prob_prime` | `probPrime($reps = 10)` | `(int): int` | probabilistic primality test (Miller-Rabin) | |
||||
| `gmp_nextprime` | `nextPrime()` | `(): BigInt` | next prime | |
||||
| `gmp_binomial` | `binomial($k)` | `(int): BigInt` | binomial coefficient C(n, k) | |
||||
| `gmp_fact` | `fact()` | `(): BigInt` | factorial n! | |
||||
|
||||
### Medium priority — advanced number-theory functions |
||||
|
||||
| GMP function | Suggested method name | Signature | Description | |
||||
|----------|-----------|------|------| |
||||
| `gmp_gcdext` | `gcdext($x)` | `(BigInt): array` | extended GCD, returns [g, s, t] such that g = s·a + t·b | |
||||
| `gmp_invert` | `invert($mod)` | `(BigInt): BigInt\|false` | modular inverse, returns false when it does not exist | |
||||
| `gmp_sqrtrem` | `sqrtrem()` | `(): array` | square root + remainder, returns [root, rem] | |
||||
| `gmp_jacobi` | `jacobi($x)` | `(BigInt): int` | Jacobi symbol | |
||||
| `gmp_legendre` | `legendre($x)` | `(BigInt): int` | Legendre symbol | |
||||
| `gmp_kronecker` | `kronecker($x)` | `(BigInt): int` | Kronecker symbol | |
||||
|
||||
### Low priority — less commonly used |
||||
|
||||
| GMP function | Suggested method name | Signature | Description | |
||||
|----------|-----------|------|------| |
||||
| `gmp_divexact` | `divExact($x)` | `(BigInt): BigInt` | exact division (used when divisibility is known; faster than ordinary division) | |
||||
| `gmp_root` | `root($n)` | `(int): BigInt` | n-th root (truncated) | |
||||
| `gmp_rootrem` | `rootrem($n)` | `(int): array` | n-th root + remainder | |
||||
| `gmp_hamdist` | `hamDist($x)` | `(BigInt): int` | Hamming distance | |
||||
|
||||
### Not applicable — conflicts with the immutable design |
||||
|
||||
| GMP function | Reason | |
||||
|----------|------| |
||||
| `gmp_setbit` | directly modifies the GMP object; BigInt is immutable | |
||||
| `gmp_clrbit` | directly modifies the GMP object; BigInt is immutable | |
||||
|
||||
### To be evaluated |
||||
|
||||
| GMP function | Description | |
||||
|----------|------| |
||||
| `gmp_scan0` | finds the first 0 bit from the specified position | |
||||
| `gmp_scan1` | finds the first 1 bit from the specified position | |
||||
| `gmp_random_bits` | generates a random-bit BigInt (needs a global seed; not suitable as an instance method) | |
||||
| `gmp_random_range` | random BigInt in a range (needs a global seed; not suitable as an instance method) | |
||||
| `gmp_random_seed` | sets the random seed (global state; not suitable as an instance method) | |
||||
| `gmp_import` | imports from a binary string | |
||||
| `gmp_export` | exports to a binary string | |
||||
|
||||
## toString enhancement |
||||
|
||||
| Missing feature | Description | |
||||
|---------|------| |
||||
| `toString($base)` | the current `toString()` only supports decimal. GMP's `gmp_strval` supports base 2-62 output | |
||||
|
||||
## Update log |
||||
|
||||
- 2026-05-27: initial version, compared against PHP 8.4.14 GMP extension |
||||
@ -0,0 +1,766 @@ |
||||
# AOT Compiler High-Precision Types Tutorial |
||||
|
||||
This tutorial introduces the three high-precision numeric types in the AOT compiler — **BigInt** (arbitrary-precision integer), **Decimal** (50-digit decimal number), and **BigFloat** (256-bit floating-point number). |
||||
|
||||
## Table of Contents |
||||
|
||||
1. [Why High-Precision Types Are Needed](#1-why-high-precision-types-are-needed) |
||||
2. [Quick Start](#2-quick-start) |
||||
3. [Overview of the Three Types](#3-overview-of-the-three-types) |
||||
4. [Construction and Declaration](#4-construction-and-declaration) |
||||
5. [Arithmetic Operations](#5-arithmetic-operations) |
||||
6. [Comparison Operations](#6-comparison-operations) |
||||
7. [Compound Assignment](#7-compound-assignment) |
||||
8. [Universal Method Calls](#8-universal-method-calls) |
||||
9. [Type Conversion](#9-type-conversion) |
||||
10. [Mixed Operations and Type Promotion](#10-mixed-operations-and-type-promotion) |
||||
11. [Automatic Detection of Extra-Long Literals](#11-automatic-detection-of-extra-long-literals) |
||||
12. [Limitations and Notes](#12-limitations-and-notes) |
||||
13. [Complete Examples](#13-complete-examples) |
||||
|
||||
--- |
||||
|
||||
## 1. Why High-Precision Types Are Needed |
||||
|
||||
PHP's native `int` is a 64-bit signed integer with a maximum value of `9223372036854775807` (about 9.22×10¹⁸). Integer literals exceeding this range are silently converted by the PHP parser to `float` (double), losing significant digits. |
||||
|
||||
PHP's native `float` (IEEE 754 double) can only guarantee about 15–16 significant digits at most. This is far from sufficient for financial computation, scientific computing, cryptography, and other scenarios. |
||||
|
||||
```php |
||||
// Precision problems with native PHP behavior |
||||
$a = 123456789012345678901234567890; // 30-digit integer → converted to float, precision lost |
||||
// Actually stored: 1.2345678901234568E+29, the trailing digits are already unreliable |
||||
|
||||
$b = 0.1 + 0.2; // 0.30000000000000004 — the classic floating-point error |
||||
``` |
||||
|
||||
The AOT compiler provides three high-precision types, built on mature C/C++ math libraries, and directly generates native calls. Here "zero-cost abstraction" refers to the absence of PHP method lookup and interpreter dispatch overhead; the high-precision operations themselves still require math library computation, memory allocation, and boxing: |
||||
|
||||
| Type | Underlying library | Characteristics | |
||||
|------|--------|------| |
||||
| BigInt | GMP (`libgmp`) | Arbitrary-precision integer, never overflows | |
||||
| Decimal | libmpdec | Decimal fraction, about 50 significant digits, no binary floating-point error | |
||||
| BigFloat | MPFR (`libmpfr`) | 256 bit by default, 64 significant digits in string output | |
||||
|
||||
--- |
||||
|
||||
## 2. Quick Start |
||||
|
||||
Prerequisites for using high-precision types: |
||||
|
||||
1. Declare `declare(strict_types=1)` at the top of the file |
||||
2. Import the native type declaration `use native_types` |
||||
3. The system must have the corresponding C++ libraries installed (`libgmp-dev`, `libmpdec-dev`, `libmpfr-dev`) |
||||
|
||||
```php |
||||
<?php |
||||
declare(strict_types=1); |
||||
use native_types; |
||||
|
||||
function main(): void { |
||||
// Your high-precision computation code |
||||
$a = std::bigInt("123456789012345678901234567890"); |
||||
$b = std::bigInt("987654321098765432109876543210"); |
||||
$sum = $a + $b; |
||||
echo $sum->toString(); |
||||
} |
||||
?> |
||||
``` |
||||
|
||||
Compile and run: |
||||
|
||||
```bash |
||||
php bin/tpc.php my_program.php -o my_program |
||||
./my_program |
||||
``` |
||||
|
||||
> **Tip**: Like all native_types, the Big* types can only be used in AOT compile mode and cannot run in the normal PHP interpreter. The AOT compiler performs compile-time evaluation of functions such as `std::bigInt()` and directly generates C++ code. |
||||
|
||||
--- |
||||
|
||||
## 3. Overview of the Three Types |
||||
|
||||
### BigInt — Arbitrary-Precision Integer |
||||
|
||||
Suitable for large integer computation; it never overflows and never loses precision. Integer division produces an integer result (truncated). |
||||
|
||||
```php |
||||
$a = std::bigInt("1234567890123456789012345678901234567890"); // 40 digits |
||||
$b = $a * 2; // 80 digits, never overflows |
||||
``` |
||||
|
||||
### Decimal — 50-Digit Decimal Number |
||||
|
||||
Suitable for scenarios requiring precise decimal representation, such as financial computation. `0.1 + 0.2` exactly equals `0.3` with no binary floating-point error. |
||||
|
||||
```php |
||||
$price = std::decimal("19.99"); |
||||
$quantity = 3; |
||||
$total = $price * $quantity; // 59.97, exact |
||||
``` |
||||
|
||||
### BigFloat — 256-Bit High-Precision Floating-Point Number |
||||
|
||||
Suitable for scenarios requiring high-precision floating-point computation, such as scientific computing. Based on MPFR, the default precision is currently fixed at 256 bit, far higher than the 53 bit of IEEE 754 double. |
||||
|
||||
```php |
||||
$pi = std::bigFloat("3.141592653589793238462643383279502884197"); |
||||
$area = $pi * 100 * 100; // high-precision π × r² |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 4. Construction and Declaration |
||||
|
||||
### 4.1 Constructing from Literals |
||||
|
||||
`std::bigInt()`, `std::decimal()`, and `std::bigFloat()` are **compile-time functions** that directly construct the corresponding C++ objects in the generated C++ code, without producing runtime function calls. |
||||
|
||||
```php |
||||
// BigInt — construct from an int or a string |
||||
$a = std::bigInt(100); // ordinary integer |
||||
$b = std::bigInt("123456789012345678901234567890"); // extra-long integer, must use a string |
||||
|
||||
// Decimal — construct from a string is recommended to avoid floating-point precision loss |
||||
$c = std::decimal("123.456"); // ✅ recommended: exact string |
||||
$d = std::decimal(42); // ✅ acceptable: from int |
||||
|
||||
// BigFloat — construct from an int, float, or string |
||||
$e = std::bigFloat(100.5); // from float |
||||
$f = std::bigFloat(42); // from int |
||||
$g = std::bigFloat("3.14159265358979323846"); // from string (exact) |
||||
``` |
||||
|
||||
### 4.2 Type Annotation |
||||
|
||||
Under `use native_types`, Big* type variables automatically get native C++ storage types: |
||||
|
||||
```php |
||||
use native_types; |
||||
|
||||
// The compiler automatically infers the type as php::BigInt / php::Decimal / php::BigFloat |
||||
$a = std::bigInt(100); // → C++: php::Variant(new BigInt(100)) |
||||
$b = std::decimal("100.50"); // → C++: php::Variant(new Decimal("100.50")) |
||||
$c = std::bigFloat(3.14); // → C++: php::Variant(new BigFloat(3.14)) |
||||
``` |
||||
|
||||
> **Key detail**: Big* types are **immutable**. Every operation returns a new value and never modifies the original variable. See [Section 7: Compound Assignment](#7-compound-assignment) for details. |
||||
|
||||
--- |
||||
|
||||
## 5. Arithmetic Operations |
||||
|
||||
### 5.1 Standard Operators |
||||
|
||||
The supported operators depend on the concrete type: BigInt supports `+ - * / % **`, Decimal supports `+ - * / %`, and BigFloat supports `+ - * /`: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$b = std::bigInt(200); |
||||
|
||||
$sum = $a + $b; // addition |
||||
$diff = $a - $b; // subtraction |
||||
$prod = $a * $b; // multiplication |
||||
$quot = $a / $b; // division (integer division for BigInt) |
||||
$mod = $a % $b; // modulo |
||||
$pow = $a ** 10; // exponentiation (supported by BigInt) |
||||
|
||||
// unary negation |
||||
$neg = -$a; // negation |
||||
``` |
||||
|
||||
Example of the generated C++ code (`$a + $b`): |
||||
|
||||
```cpp |
||||
php::BigInt::add(a, b) // BigInt addition |
||||
php::BigInt::sub(a, b) // BigInt subtraction |
||||
php::BigInt::mul(a, b) // BigInt multiplication |
||||
php::BigInt::div(a, b) // BigInt division |
||||
php::BigInt::mod(a, b) // BigInt modulo |
||||
php::BigInt::pow(a, b) // BigInt exponentiation |
||||
``` |
||||
|
||||
### 5.2 Mixed Operations with int / float |
||||
|
||||
Big* types can be mixed with ordinary int/float within a safe range, and the compiler automatically performs type promotion: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
|
||||
$b = $a + 50; // BigInt + Int → BigInt |
||||
$c = 200 + $a; // Int + BigInt → BigInt |
||||
$d = $a * 3.5; // BigInt * Float → compile error! |
||||
// a float cannot be promoted to BigInt exactly, |
||||
// use Decimal or BigFloat instead |
||||
``` |
||||
|
||||
### 5.3 BigInt Division Notes |
||||
|
||||
`BigInt / BigInt` is integer division (truncation), similar to PHP's `intdiv()`: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$b = $a / 3; // 33 (not 33.333...) |
||||
``` |
||||
|
||||
If you need an exact decimal result, convert the operands to Decimal first: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$result = std::decimal($a->toString()) / std::decimal("3"); |
||||
// 33.333333333... |
||||
``` |
||||
|
||||
### 5.4 Summary of Operators Supported by Each Type |
||||
|
||||
| Operator | BigInt | Decimal | BigFloat | |
||||
|--------|--------|---------|----------| |
||||
| `+` `-` `*` | ✅ | ✅ | ✅ | |
||||
| `/` | ✅ integer division | ✅ | ✅ | |
||||
| `%` | ✅ | ✅ | ❌ | |
||||
| `**` | ✅ | ❌ | ❌ | |
||||
| `-` (unary negation) | ✅ | ✅ | ✅ | |
||||
|
||||
--- |
||||
|
||||
## 6. Comparison Operations |
||||
|
||||
All six comparison operators can be used with Big* types: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$b = 200; |
||||
|
||||
// comparison operations return bool (an (int) cast is needed for output) |
||||
echo (int)($a < $b); // 1 (true) → cmp(a,b) < 0 |
||||
echo (int)($a > $b); // 0 (false) → cmp(a,b) > 0 |
||||
echo (int)($a <= 100); // 1 (true) → cmp(a,b) <= 0 |
||||
echo (int)($a >= 100); // 1 (true) → cmp(a,b) >= 0 |
||||
echo (int)($a == 100); // 1 (true) → cmp(a,b) == 0 |
||||
echo (int)($a != 50); // 1 (true) → cmp(a,b) != 0 |
||||
|
||||
// spaceship operator |
||||
$cmp = $a <=> $b; // -1 ($a < $b) |
||||
echo (int)$cmp; // -1 |
||||
``` |
||||
|
||||
Example of the generated C++ code: |
||||
|
||||
```cpp |
||||
php::BigInt::cmp(a, b) < 0 // a < b |
||||
php::BigInt::cmp(a, b) == 0 // a == b |
||||
php::BigInt::cmp(a, b) != 0 // a != b |
||||
php::BigInt::cmp(a, b) // a <=> b (directly returns -1/0/1) |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 7. Compound Assignment |
||||
|
||||
Big* types **support** compound assignment operators such as `+=`, `-=`, `*=`, `/=`, and `%=`. |
||||
|
||||
### 7.1 How It Works |
||||
|
||||
Big* types are **immutable**. `$a += 50` is expanded at compile time to `$a = BigInt::add($a, 50)` — a new value is created and then assigned to the original variable. |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$a += 50; // → a = php::BigInt::add(a, php::newBigInt(50)) |
||||
echo $a->toString(); // "150" |
||||
|
||||
$a -= 30; // → a = php::BigInt::sub(a, php::newBigInt(30)) |
||||
$a *= 5; // → a = php::BigInt::mul(a, php::newBigInt(5)) |
||||
$a /= 3; // → a = php::BigInt::div(a, php::newBigInt(3)) |
||||
$a %= 7; // → a = php::BigInt::mod(a, php::newBigInt(7)) |
||||
``` |
||||
|
||||
Decimal and BigFloat support it likewise: |
||||
|
||||
```php |
||||
// Decimal compound assignment |
||||
$d = std::decimal("100.50"); |
||||
$d += 25.25; // → d = php::Decimal::add(d, php::newDecimal("25.25")) |
||||
$d -= 123.45; // → d = php::Decimal::sub(d, php::newDecimal("123.45")) |
||||
$d *= 2; // → d = php::Decimal::mul(d, php::newDecimal(2)) |
||||
$d /= 4; // → d = php::Decimal::div(d, php::newDecimal(4)) |
||||
$d %= 5.0; // → d = php::Decimal::mod(d, php::newDecimal("5.0")) |
||||
|
||||
// BigFloat compound assignment (% is not supported) |
||||
$bf = std::bigFloat(100.0); |
||||
$bf += 50.0; |
||||
$bf -= 30.0; |
||||
$bf *= 2.0; |
||||
$bf /= 3.0; |
||||
``` |
||||
|
||||
### 7.2 `++` / `--` Are Unavailable |
||||
|
||||
Because Big* types are immutable, the `++` / `--` operators do not match semantically. The compiler gives a clear error message: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$a++; // ❌ compile error: Cannot use ++ on php::BigInt. Use += 1 instead. |
||||
++$a; // ❌ compile error: Cannot use ++ on php::BigInt. Use += 1 instead. |
||||
--$a; // ❌ compile error: Cannot use -- on php::BigInt. Use -= 1 instead. |
||||
``` |
||||
|
||||
The correct alternatives: |
||||
|
||||
```php |
||||
$a += 1; // ✅ instead of $a++ |
||||
$a -= 1; // ✅ instead of $a-- |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 8. Universal Method Calls |
||||
|
||||
Big* types support calling methods via the `$value->method()` syntax (Universal Methods). These calls are directly translated at compile time to the corresponding C++ static functions, with no dynamic method dispatch overhead; the math library computation, result allocation, and boxing costs still remain. |
||||
|
||||
### 8.1 BigInt Methods |
||||
|
||||
```php |
||||
$a = std::bigInt("12345678901234567890"); |
||||
|
||||
// arithmetic methods (all return a new BigInt) |
||||
$b = $a->add(1); // addition: $a + 1 |
||||
$c = $a->sub(1); // subtraction: $a - 1 |
||||
$d = $a->mul(2); // multiplication: $a * 2 |
||||
$e = $a->div(10); // division: $a / 10 |
||||
$f = $a->mod(1000000); // modulo: $a % 1000000 |
||||
$g = $a->pow(3); // exponentiation: $a ** 3 |
||||
|
||||
// unary methods |
||||
$h = $a->neg(); // negation: -$a |
||||
$i = $a->abs(); // absolute value |
||||
|
||||
// special methods |
||||
$j = $a->gcd(15); // greatest common divisor: gcd($a, 15) |
||||
|
||||
// comparison methods |
||||
$cmp = $a->cmp(100); // comparison: returns -1/0/1 |
||||
if ($a->cmp(100) > 0) { /* $a > 100 */ } |
||||
|
||||
// type conversion methods |
||||
echo $a->toString(); // to string: "12345678901234567890" |
||||
echo $a->toInt(); // to int; throws ArithmeticError when out of the PHP int range |
||||
echo $a->toFloat(); // to float (may lose precision) |
||||
``` |
||||
|
||||
### 8.2 Decimal Methods |
||||
|
||||
```php |
||||
$d = std::decimal("123.456"); |
||||
|
||||
// arithmetic methods |
||||
echo $d->add(std::decimal("50.25"))->toString(); // "173.706" |
||||
echo $d->sub(std::decimal("50.25"))->toString(); // "73.206" |
||||
echo $d->mul(2)->toString(); // "246.912" |
||||
echo $d->div(3)->toString(); // "41.152" |
||||
echo $d->mod(std::decimal("5.0"))->toString(); // "3.456" |
||||
|
||||
// unary methods |
||||
echo $d->neg()->toString(); // "-123.456" |
||||
echo $d->abs()->toString(); // "123.456" |
||||
|
||||
// comparison and conversion |
||||
echo $d->cmp(std::decimal("100")) > 0 ? "greater" : "less"; // "greater" |
||||
echo $d->toInt(); // 123 |
||||
echo $d->toString(); // "123.456" |
||||
``` |
||||
|
||||
### 8.3 BigFloat Methods |
||||
|
||||
```php |
||||
$bf = std::bigFloat(3.14159265); |
||||
|
||||
echo $bf->add(1.0)->toString(); // "4.14159265..." |
||||
echo $bf->mul(2.0)->toString(); // "6.2831853..." |
||||
echo $bf->div(2.0)->toFloat(); // 1.570796325 |
||||
echo $bf->neg()->toString(); // "-3.14159265..." |
||||
echo $bf->abs()->toString(); // "3.14159265..." |
||||
|
||||
// comparison |
||||
echo $bf->cmp(3.0); // > 0 ($bf > 3.0) |
||||
``` |
||||
|
||||
### 8.4 Universal Methods vs Operators |
||||
|
||||
Operators and method calls are functionally equivalent; which one to choose depends on coding style: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$b = std::bigInt(50); |
||||
|
||||
// two equivalent ways of writing |
||||
$result1 = $a + $b; // operator style |
||||
$result2 = $a->add($b); // method call style |
||||
|
||||
// method calls support chaining |
||||
$result3 = $a->add(10)->mul(2)->sub(5)->toString(); // "215" |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 9. Type Conversion |
||||
|
||||
### 9.1 Conversion Between Big* Types |
||||
|
||||
```php |
||||
// BigInt → Decimal (exact, recommended approach) |
||||
$big = std::bigInt("12345678901234567890"); |
||||
$dec = std::decimal($big->toString()); |
||||
|
||||
// Decimal → BigInt (truncates the fractional part) |
||||
$d = std::decimal("123.456"); |
||||
$i = std::bigInt($d->toInt()); // 123 |
||||
|
||||
// Int → BigInt / Decimal / BigFloat |
||||
$bi = std::bigInt(42); |
||||
$dc = std::decimal(42); |
||||
$bf = std::bigFloat(42); |
||||
|
||||
// Float → BigFloat (using a float literal directly for Float → Decimal is not recommended) |
||||
$bf2 = std::bigFloat(3.14); |
||||
|
||||
// any type → BigFloat |
||||
$bf3 = std::bigFloat($big->toString()); |
||||
``` |
||||
|
||||
### 9.2 Conversion Between Big* and Ordinary Types |
||||
|
||||
```php |
||||
// BigInt → ordinary types |
||||
$a = std::bigInt("99999999999999999999"); |
||||
$s = $a->toString(); // "99999999999999999999" |
||||
$i = $a->toInt(); // throws ArithmeticError when out of the PHP int range |
||||
$f = $a->toFloat(); // 1.0E+20 (may lose precision) |
||||
|
||||
// ordinary types → BigInt (via compile-time functions) |
||||
$b = std::bigInt(42); // int → BigInt |
||||
$c = std::bigInt("123456..."); // string → BigInt |
||||
|
||||
// explicit casts and PHP conversion functions convert numerically and do not read the Box resource id |
||||
$n = (int) std::decimal("12.75"); // 12 |
||||
$x = floatval(std::bigInt("42")); // 42.0 |
||||
$ok = boolval(std::bigFloat("0")); // false |
||||
``` |
||||
|
||||
### 9.3 Limitations on Cross-Type Implicit Mixing |
||||
|
||||
The compiler blocks cross-type implicit mixing operations that may cause precision loss: |
||||
|
||||
```php |
||||
$a = std::bigFloat(100.5); |
||||
$b = std::bigInt(200); |
||||
|
||||
$c = $a + $b; // ❌ compile error: Cannot mix BigFloat and BigInt implicitly. |
||||
// Use std::bigFloat() to convert explicitly. |
||||
|
||||
// the correct approach: explicit conversion |
||||
$c = $a + std::bigFloat($b->toString()); // ✅ |
||||
``` |
||||
|
||||
| Combination | Allowed | Description | |
||||
|------|---------|------| |
||||
| BigInt + BigFloat | ❌ compile error | different precision metrics, explicit conversion required | |
||||
| BigInt + Decimal | ❌ compile error | different precision metrics, explicit conversion required | |
||||
| BigFloat + Decimal | ❌ compile error | different precision metrics, explicit conversion required | |
||||
| BigInt + Int | ✅ automatically promote Int → BigInt | no precision loss | |
||||
| BigInt + Float | ❌ compile error | Float cannot be promoted to BigInt exactly | |
||||
| Decimal + Int | ✅ automatically promote Int → Decimal | no precision loss | |
||||
| Decimal + Float | ✅ automatically promote Float → Decimal | may have a tiny error | |
||||
| BigFloat + Int | ✅ automatically promote Int → BigFloat | no precision loss | |
||||
| BigFloat + Float | ✅ automatically promote Float → BigFloat | no precision loss | |
||||
|
||||
--- |
||||
|
||||
## 10. Mixed Operations and Type Promotion |
||||
|
||||
When Big* types are mixed with ordinary Int/Float, the compiler only performs safe promotions that do not change the numeric model. |
||||
|
||||
**Rules**: |
||||
|
||||
1. If either operand is a Var (non-native type), both are converted to Var and runtime computation uses the ZendVM |
||||
2. If both operands are Int/Float, Float takes precedence (Int → Float) |
||||
3. BigInt can safely promote Int; Decimal can promote Int and Float literals whose source text is preserved; BigFloat can promote Int/Float |
||||
4. No implicit conversion is performed between different Big* types, or between BigInt and Float |
||||
|
||||
```php |
||||
// type promotion examples |
||||
$a = std::bigInt(100); |
||||
$b = 50; // Int |
||||
|
||||
$c = $a + $b; // BigInt + Int → BigInt |
||||
// $b is automatically promoted to BigInt |
||||
|
||||
$d = std::decimal("10.5"); |
||||
$e = $d + 3; // Decimal + Int → Decimal |
||||
// 3 is automatically promoted to Decimal |
||||
|
||||
$f = std::bigFloat(1.5); |
||||
$g = $f + 2.0; // BigFloat + Float → BigFloat |
||||
// 2.0 is automatically promoted to BigFloat |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 11. Automatic Detection of Extra-Long Literals |
||||
|
||||
The AOT compiler automatically detects numeric literals that exceed the precision of native types and automatically converts them to the corresponding Big* type. You do **not need to wrap them manually**. |
||||
|
||||
```php |
||||
// integer with 19 or more digits → automatically converted to BigInt |
||||
$a = 12345678901234567890; |
||||
echo $a->toString(); // "12345678901234567890" |
||||
// the compiler handles it automatically: equivalent to std::bigInt("12345678901234567890") |
||||
|
||||
// decimal with 16 or more significant digits → automatically converted to Decimal |
||||
$b = 3.14159265358979323846; |
||||
// the compiler handles it automatically: equivalent to std::decimal("3.14159265358979323846") |
||||
``` |
||||
|
||||
**Detection rules**: |
||||
|
||||
- pure digits, 19 or more digits → BigInt |
||||
- contains a decimal point or exponent, 16 or more significant digits → Decimal |
||||
- underscores `_` are disabled (e.g. `1_234_567_890_123_456_789_0`) |
||||
|
||||
> **Recommended practice**: For critical precision, it is still recommended to explicitly use `std::bigInt("...")` or `std::decimal("...")` to ensure clear intent. Automatic detection is a convenience feature suited for rapid prototyping. |
||||
|
||||
--- |
||||
|
||||
## 12. Limitations and Notes |
||||
|
||||
### 12.1 Immutability |
||||
|
||||
All Big* types are **immutable**. Every operation creates a new value: |
||||
|
||||
```php |
||||
$a = std::bigInt(100); |
||||
$b = $a->add(50); // $a is still 100, $b is 150 |
||||
$c = $a + 50; // $a is still 100, $c is 150 |
||||
``` |
||||
|
||||
### 12.2 `++` / `--` Not Supported |
||||
|
||||
See [Section 7.2](#72---are-unavailable). Use `+= 1` / `-= 1` instead. |
||||
|
||||
### 12.3 BigFloat Does Not Support `%` and `**` |
||||
|
||||
```php |
||||
$bf = std::bigFloat(10.0); |
||||
$bf %= 3; // ❌ compile error |
||||
$bf ** 2; // ❌ compile error |
||||
``` |
||||
|
||||
### 12.4 Decimal Does Not Support `**` |
||||
|
||||
```php |
||||
$d = std::decimal("10.5"); |
||||
$d ** 2; // ❌ compile error |
||||
``` |
||||
|
||||
### 12.5 Cross Big* Types Cannot Be Implicitly Mixed |
||||
|
||||
BigFloat, Decimal, and BigInt must be explicitly converted: |
||||
|
||||
```php |
||||
$a = std::bigFloat(100.5); |
||||
$b = std::bigInt(200); |
||||
$c = $a + $b; // ❌ compile error |
||||
// change to |
||||
$c = $a + std::bigFloat($b->toString()); // ✅ |
||||
``` |
||||
|
||||
This restriction also applies to comparison operations. Before comparing, both sides must be explicitly converted to the same Big* type to avoid compiling to the wrong underlying resource type. |
||||
|
||||
### 12.6 Boundaries and Exceptions |
||||
|
||||
- BigInt negative right shifts use arithmetic right shift, for example `std::bigInt("-3") >> 1` yields `-2`. |
||||
- A negative bit index, negative `popCount()`, or an excessively large exponent throws `ValueError`. |
||||
- Division by zero throws `DivisionByZeroError`; converting to a PHP int beyond range throws `ArithmeticError`. |
||||
- When the absolute value of a BigFloat's exponent exceeds 10000, `toString()` automatically uses scientific notation to avoid constructing an excessively large string. |
||||
|
||||
### 12.7 Cannot Run in the Normal PHP Interpreter |
||||
|
||||
Big* types are a proprietary feature of the AOT compiler, relying on compile-time code generation and C++ underlying libraries. The source code cannot be directly interpreted and executed by the `php` command. |
||||
|
||||
### 12.8 Enabling `use native_types` |
||||
|
||||
Forgetting to add `use native_types` causes Big* variables to be treated as Var (generic type), losing most of the performance advantages of native types. |
||||
|
||||
--- |
||||
|
||||
## 13. Complete Examples |
||||
|
||||
### 13.1 Large Integer Factorial |
||||
|
||||
```php |
||||
<?php |
||||
declare(strict_types=1); |
||||
use native_types; |
||||
|
||||
/** |
||||
* Compute the factorial of n, supporting arbitrarily large results |
||||
*/ |
||||
function factorial(int $n): void { |
||||
$result = std::bigInt(1); |
||||
for ($i = 2; $i <= $n; $i++) { |
||||
$result *= $i; |
||||
} |
||||
echo "{$n}! = " . $result->toString() . "\n"; |
||||
echo "digits: " . strlen($result->toString()) . "\n"; |
||||
} |
||||
|
||||
function main(): void { |
||||
factorial(10); // 10! = 3628800 |
||||
factorial(50); // 3041409320171337804361260816606476884... |
||||
factorial(100); // 933262154439441526816992388562667004... |
||||
} |
||||
?> |
||||
``` |
||||
|
||||
### 13.2 Financial Computation: Order Details |
||||
|
||||
```php |
||||
<?php |
||||
declare(strict_types=1); |
||||
use native_types; |
||||
|
||||
function main(): void { |
||||
// use Decimal to represent amounts exactly |
||||
$price = std::decimal("19.99"); |
||||
$quantity = 3; |
||||
$taxRate = std::decimal("0.08"); |
||||
|
||||
$subtotal = $price * $quantity; |
||||
$tax = $subtotal * $taxRate; |
||||
$total = $subtotal + $tax; |
||||
|
||||
echo "unit price: " . $price->toString() . "\n"; |
||||
echo "quantity: {$quantity}\n"; |
||||
echo "subtotal: " . $subtotal->toString() . "\n"; |
||||
echo "tax: " . $tax->toString() . "\n"; |
||||
echo "total: " . $total->toString() . "\n"; |
||||
} |
||||
?> |
||||
``` |
||||
|
||||
Output: |
||||
|
||||
``` |
||||
unit price: 19.99 |
||||
quantity: 3 |
||||
subtotal: 59.97 |
||||
tax: 4.7976 |
||||
total: 64.7676 |
||||
``` |
||||
|
||||
### 13.3 High-Precision Pi Computation |
||||
|
||||
```php |
||||
<?php |
||||
declare(strict_types=1); |
||||
use native_types; |
||||
|
||||
function main(): void { |
||||
// use BigFloat for high-precision math computation |
||||
$pi = std::bigFloat("3.141592653589793238462643383279502884197"); |
||||
$radius = 100; |
||||
|
||||
// area of a circle |
||||
$area = $pi * std::bigFloat($radius * $radius); |
||||
echo "circle area: " . $area->toString() . "\n"; |
||||
|
||||
// circumference of a circle |
||||
$circumference = $pi * std::bigFloat(2 * $radius); |
||||
echo "circumference: " . $circumference->toString() . "\n"; |
||||
|
||||
// comparison |
||||
$earthRadius = 6371; |
||||
$earthArea = $pi * std::bigFloat($earthRadius * $earthRadius); |
||||
echo "if the radius is {$earthRadius}km...\n"; |
||||
echo "approximate area: " . $earthArea->toInt() . " km²\n"; |
||||
} |
||||
?> |
||||
``` |
||||
|
||||
### 13.4 Comprehensive Example: Mixing Multiple Types |
||||
|
||||
```php |
||||
<?php |
||||
declare(strict_types=1); |
||||
use native_types; |
||||
|
||||
function main(): void { |
||||
// BigInt — large integer operations |
||||
$big = std::bigInt("100000000000000000000"); |
||||
$big += std::bigInt("99999999999999999999"); |
||||
echo "BigInt: " . $big->toString() . "\n"; |
||||
|
||||
// operators + comparison |
||||
$a = std::bigInt(100); |
||||
echo "BigInt + Int: " . ($a + 50)->toString() . "\n"; |
||||
echo "BigInt * 5: " . ($a * 5)->toString() . "\n"; |
||||
echo "BigInt > 50: " . (int)($a > 50) . "\n"; |
||||
echo "a == 100: " . (int)($a == 100) . "\n"; |
||||
|
||||
// Unary minus |
||||
$neg = -$a; |
||||
echo "-a: " . $neg->toString() . "\n"; |
||||
|
||||
// Decimal — exact decimal operations |
||||
$price = std::decimal("99.99"); |
||||
$price *= 3; // compound assignment |
||||
echo "price × 3: " . $price->toString() . "\n"; |
||||
|
||||
// comparison |
||||
$d = std::decimal("100.25"); |
||||
echo "d > 50: " . (int)($d > 50) . "\n"; |
||||
echo "d != 100: " . (int)($d != 100) . "\n"; |
||||
|
||||
// BigFloat — high-precision floating point |
||||
$bf = std::bigFloat(3.14159); |
||||
$bf *= 2.0; |
||||
echo "pi × 2: " . $bf->toString() . "\n"; |
||||
|
||||
// method chaining |
||||
$result = std::bigInt(100) |
||||
->add(50) |
||||
->mul(3) |
||||
->sub(100) |
||||
->toString(); |
||||
echo "100 + 50 × 3 - 100 = " . $result . "\n"; |
||||
} |
||||
?> |
||||
``` |
||||
|
||||
Output: |
||||
|
||||
``` |
||||
BigInt: 200000000000000000099 |
||||
BigInt + Int: 150 |
||||
BigInt * 5: 500 |
||||
BigInt > 50: 1 |
||||
a == 100: 1 |
||||
-a: -100 |
||||
price × 3: 299.97 |
||||
d > 50: 1 |
||||
d != 100: 1 |
||||
pi × 2: 6.2831800000000000 |
||||
100 + 50 × 3 - 100 = 350 |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Further Reading |
||||
|
||||
- **Type System Specification**: [`docs/NATIVE_TYPES.md`](NATIVE_TYPES.md) — complete type promotion rules, declaration syntax, and C++ API reference |
||||
- **BigInt PHPT Tests**: [`tests/compiler/bigint/`](../tests/compiler/bigint/) — integration tests for BigInt features |
||||
- **Decimal PHPT Tests**: [`tests/compiler/decimal/`](../tests/compiler/decimal/) — integration tests for Decimal features |
||||
- **BigFloat Integration Tests**: [`tests/compiler/bignumber/bigfloat_operators.phpt`](../tests/compiler/bignumber/bigfloat_operators.phpt) — BigFloat operator tests |
||||
- **C++ Runtime Header Files**: |
||||
- [`phpx/include/phpx_big_int.h`](../../phpx/include/phpx_big_int.h) — BigInt C++ API |
||||
- [`phpx/include/phpx_decimal.h`](../../phpx/include/phpx_decimal.h) — Decimal C++ API |
||||
- [`phpx/include/phpx_big_float.h`](../../phpx/include/phpx_big_float.h) — BigFloat C++ API |
||||
@ -0,0 +1,114 @@ |
||||
# `#[Immutable]` compile-time effect checking |
||||
|
||||
## Purpose |
||||
|
||||
`#[Immutable]` is a TypePHP compile-time annotation modelled after C++ `const`. |
||||
It prevents accidental mutation in statically compiled code without adding a |
||||
wrapper object, Zend metadata, runtime branch, or ABI change. |
||||
|
||||
It is intentionally a best-effort static tool rather than a security boundary. |
||||
Calls whose target is deliberately made dynamic are an escape hatch and do not |
||||
receive a runtime guard. |
||||
|
||||
## Supported targets |
||||
|
||||
```php |
||||
#[Immutable] |
||||
public function name(): string |
||||
{ |
||||
return $this->name; |
||||
} |
||||
|
||||
function inspect(#[Immutable] User $user): string |
||||
{ |
||||
return $user->name(); |
||||
} |
||||
``` |
||||
|
||||
The attribute is valid on methods and on function, method, and closure |
||||
parameters. On an instance method it makes `$this` immutable. On a parameter it |
||||
makes the binding immutable and, when it can contain an object, treats the |
||||
referenced object as immutable as well. |
||||
|
||||
## Rejected operations |
||||
|
||||
For an immutable root such as `$this` or `$user`, the compiler rejects: |
||||
|
||||
- assignment, destructuring, and array-element or object-property writes; |
||||
- compound assignment, `++`, `--`, `unset()`, taking a reference, and |
||||
`foreach (... as &$value)`; |
||||
- a statically named method call unless the resolved method is also marked |
||||
`#[Immutable]`; |
||||
- a mutating value extension such as `$array->sort()`; read-only array/string |
||||
methods remain available; |
||||
- passing an object to a statically resolved parameter that is not itself |
||||
`#[Immutable]`; |
||||
- passing any immutable value to a mutable by-reference parameter, including |
||||
extension functions such as `sort()`; |
||||
- storing an immutable object identity in an object property, array, |
||||
global/static variable, or returning/yielding it as a mutable value. |
||||
|
||||
An immutable by-reference parameter is supported. It acts like a C++ `const &`: |
||||
the reference is accepted because the callee is checked against mutation. |
||||
|
||||
`#[MethodsFor]` follows the same contract. An object extension is callable on |
||||
an immutable receiver only when its receiver parameter is marked |
||||
`#[Immutable]`. |
||||
|
||||
## Aliases, closures, generators, and inheritance |
||||
|
||||
Local aliases of immutable objects remain immutable: |
||||
|
||||
```php |
||||
$alias = $user; |
||||
$alias->rename('new'); // compile-time error |
||||
``` |
||||
|
||||
`clone` creates a distinct mutable object and therefore intentionally drops the |
||||
annotation. Captured variables, arrow functions, closure `$this`, and Fiber |
||||
generator bodies carry immutable metadata into their generated function |
||||
contexts. |
||||
|
||||
An overriding class or interface method may strengthen an ordinary contract by |
||||
adding `#[Immutable]`, but it cannot remove `#[Immutable]` from an inherited |
||||
method or parameter. |
||||
|
||||
## Value versus object semantics |
||||
|
||||
Scalar values and PHP copy-on-write values can be read and copied normally. For |
||||
example, `count($values)` and `$copy = $values` do not modify an immutable array. |
||||
The compiler propagates immutability through an expression only when object |
||||
identity is possible. |
||||
|
||||
## Explicit escape hatches |
||||
|
||||
The following intentionally bypass static method-effect checking: |
||||
|
||||
```php |
||||
$method = 'rename'; |
||||
$user->$method('new'); |
||||
|
||||
$callable = getRuntimeCallable(); |
||||
$callable($user); |
||||
``` |
||||
|
||||
The same applies to other runtime-only mechanisms that hide the target from the |
||||
compiler, including reflection and dynamic ZendVM code. TypePHP neither inserts |
||||
a runtime read-only proxy nor attempts to recover the escaped value later. |
||||
|
||||
This boundary is deliberate: `#[Immutable]` should cost nothing in generated |
||||
code and should not complicate PHPX/ZendVM object semantics. |
||||
|
||||
## Property hooks and magic access |
||||
|
||||
Property-hook reads are lowered to generated method calls. Consequently, a hook |
||||
used through an immutable receiver must itself carry an `#[Immutable]` method |
||||
contract; otherwise the generated call is rejected. Fully dynamic magic access |
||||
is covered by the same escape-hatch rule as other runtime-only behavior. |
||||
|
||||
## Implementation boundaries |
||||
|
||||
The implementation is isolated in `src/Immutable/ImmutableSupportTrait.php`. |
||||
`FunctionDef` and `ArgInfo` retain only compile-time effect bits, while each |
||||
`FunctionContext` stores the immutable roots and object aliases relevant to that |
||||
body. Checks run during AST lowering and emit no C++ code when successful. |
||||
@ -0,0 +1,182 @@ |
||||
# AOT/PHP Incompatibility List |
||||
|
||||
This document lists only the key areas in which the current AOT compiler is |
||||
incompatible with or more restrictive than standard PHP. |
||||
|
||||
## Program structure |
||||
|
||||
- Executable statements are not allowed at global scope; only static constructs |
||||
such as declarations, `use`, `declare`, and constant definitions are allowed. |
||||
- Function declarations are not allowed inside functions or methods. |
||||
- Named class declarations are not allowed inside functions or methods. |
||||
- Binary mode must define a global `main()`. |
||||
- `main()` may either take no parameters or have the signature |
||||
`(int $argc, array $argv)`. |
||||
- `main()` must return `void`. |
||||
|
||||
## Declarations and types |
||||
|
||||
- Variable variables `$$var` are not supported. |
||||
- PHP 8.5 `#[NoDiscard]` is not supported yet. |
||||
- The PHP 8.5 `(void)` cast is supported for explicitly discarding a value; the |
||||
operand is still evaluated and its side effects are preserved, and the cast |
||||
cannot be used in value contexts such as assignments, returns, arguments, or |
||||
conditions. |
||||
- Support for PHP 8.5 `clone()` / clone-with requires the linked `libphp` to be |
||||
version 8.5 or later. Public and dynamic properties, private/protected/readonly |
||||
properties, property hooks, call ordering, error propagation, and callable |
||||
paths are all covered by PHPT tests. |
||||
- PHP 8.4 property hooks are compiled into AOT getters/setters and register the |
||||
corresponding Zend hook metadata; direct property reads/writes, Reflection, |
||||
and object iteration are all supported. Taking a reference to a hooked |
||||
property is currently not supported. |
||||
- Interface property hooks do not currently support an explicitly declared |
||||
`set` parameter; use the implicit setter value parameter. |
||||
- PHP 8.4 Reflection Lazy Objects cannot be used with TypePHP AOT classes. AOT |
||||
classes are registered as persistent internal classes, and Zend's |
||||
`zend_object_make_lazy()` explicitly rejects internal classes. Zend PHP user |
||||
classes loaded dynamically at runtime are not subject to this restriction. |
||||
- `private(set)` and `protected(set)` asymmetric property visibility is |
||||
supported, including constructor property promotion. Zend-backed objects |
||||
perform the scope check through the PHP 8.4+ class-level object handler and |
||||
preserve the promoted / set-visibility / implicit-final reflection flags; |
||||
Native objects enforce the equivalent scope rules through compile-time access |
||||
checks. |
||||
- Final properties declared through constructor promotion are supported, but |
||||
TypePHP requires an explicit `public`, `protected`, or `private` modifier; |
||||
PHP 8.5's implicitly public form, `final int $value`, is not accepted. As a |
||||
TypePHP extension, this syntax is independent of the PHP source-syntax version |
||||
supported by the linked `libphp` and remains available when using a PHP 8.4 |
||||
`libphp.so`. |
||||
- TypePHP forbids attributes on global or namespaced constant declarations; PHP |
||||
8.5 global constant attributes are out of scope. Class constant attributes are |
||||
not affected by this restriction. |
||||
- A `.stub.php` file only declares Zend ABI symbols supplied by external C++ and |
||||
must not use `#[Native]` on its classes. Native Class object layouts must be |
||||
generated and owned by the TypePHP compiler. |
||||
- Returning by reference from closures or arrow functions is not supported. |
||||
- PHP 8.5 `static function` expressions in global constants, class constants, |
||||
parameter defaults, or property defaults are not supported yet. Closures |
||||
nested inside initializer expressions are likewise rejected at compile time. |
||||
- `__construct()` may not have a return value. |
||||
- A parameter with a default value may not appear before a required parameter |
||||
(PHP permits this legacy pattern but treats the former parameter as required). |
||||
- By-reference variadic parameters `&...$args` are supported for ordinary |
||||
functions and methods whose signature is known at compile time, including |
||||
direct, named, and unpacked arguments. A by-reference variadic declaration on |
||||
a dynamic Closure is not supported. |
||||
- Union, intersection, and nullable types are still represented as `mixed/any` |
||||
in C++, but the static analysis phase uses known expression types to reject |
||||
definitely incompatible arguments, return values, and property assignments |
||||
ahead of time; dynamic values still retain their runtime type checks. |
||||
- Once a local variable's type has been statically inferred as a concrete native |
||||
type, reassigning it to an incompatible type within the same scope is not |
||||
supported. |
||||
|
||||
## declare |
||||
|
||||
- `declare(ticks=...)` is not supported. |
||||
- `declare(encoding=...)` accepts only `UTF-8`. |
||||
- `declare(strict_types=...)` accepts only `strict_types=1`. |
||||
- No other `declare` directives are supported. |
||||
|
||||
## Calls and references |
||||
|
||||
- `exit(message: $value)` is available as a TypePHP named-argument extension; it |
||||
enters the same exit path as the positional form `exit($value)`. |
||||
- TypePHP uses strict argument-count rules: non-variadic functions do not accept |
||||
extra arguments beyond the declared signature, and `func_get_args()` does not |
||||
implicitly relax the signature. |
||||
- Reference parameters and write-back semantics are supported for ordinary |
||||
functions, ordinary methods, and native direct calls with known signatures; |
||||
do not mistakenly describe the compiler's internal cross-trait dynamic-dispatch |
||||
limitation as "TypePHP does not support reference parameters". |
||||
- Closures and arrow functions support fixed by-reference parameters. Because a |
||||
Closure invocation is dynamically dispatched, the caller must still mark |
||||
reference arguments explicitly with `refval()` / `toRef()`; Zend callbacks |
||||
use the generated Closure arginfo automatically. |
||||
- Reference assignment cannot create a reference from a complex static-property |
||||
expression. |
||||
- Calls whose argument signature cannot be determined at compile time — dynamic |
||||
calls, closure calls, and the like — cannot convert reference parameters |
||||
automatically; `refval()` or the equivalent keyword method `toRef()` must be |
||||
used explicitly. |
||||
- `refval()` / `toRef()` only accept variables, array elements, or object |
||||
properties. |
||||
- A call that uses argument unpacking followed by named arguments falls back to |
||||
dynamic dispatch and cannot use the native call path. |
||||
|
||||
## Object model |
||||
|
||||
- Reserved keyword methods such as `toInt()`, `toString()`, and `toArray()` are |
||||
resolved before ordinary object methods; an application method of the same |
||||
name that takes arguments is not called with ordinary object-method semantics. |
||||
- `toAny()` and `toRef()` are non-overridable TypePHP keyword methods, and |
||||
ordinary class-like declarations must not define methods with these names |
||||
(method names are case-insensitive, per PHP rules). A Native class may only |
||||
explicitly define a `toAny()` conversion method returning `mixed/any`; no |
||||
implicit conversion is provided. Native classes do not support `toRef()`. |
||||
- Fixed-layout typed properties that are not explicitly initialized use the |
||||
zero value of their type and do not preserve Zend PHP's full uninitialized |
||||
state; expressions such as `??` that depend on the uninitialized state may |
||||
therefore behave differently. |
||||
- A subclass may not shadow a parent's private property with a `private` |
||||
property of the same name; `public` / `protected` declarations of the same name |
||||
are treated as the same inherited property slot and must still satisfy the |
||||
type, visibility, and `readonly` compatibility requirements. |
||||
- Dynamic writes to typed properties still use strict type checks. When the |
||||
right-hand side cannot be determined at compile time, TypePHP preserves a |
||||
runtime check rather than falling back to Zend weak scalar conversion. |
||||
- Native Classes use a separate fixed-layout object model. They cannot be used |
||||
as ordinary Zend Objects, PHP array keys/values, or arbitrary `mixed` values, |
||||
and they restrict dynamic members, references, static/readonly members, and |
||||
several operators. See [Native Class Object Design](NATIVE_CLASS_OBJECT.md) |
||||
for the complete boundary. |
||||
|
||||
## Expressions and control flow |
||||
|
||||
- A `match` arm condition may not itself be a `match` expression. |
||||
- The value target in a by-reference `foreach` may only be a variable. |
||||
- `foreach` list destructuring does not support binding elements by reference. |
||||
- On the non-`int/bool` lowering path, every non-empty `switch` case must end in |
||||
`return`, `break`, `continue`, `exit`, or `throw`; do not rely on implicit PHP |
||||
case fallthrough. The native `int/bool` switch path can currently retain C++ |
||||
fallthrough, so project code should terminate every non-empty case explicitly. |
||||
- Appending, inserting, `unset()`, and wholesale replacement of `std::vector`, |
||||
`std::map`, and `std::ordered_map` are forbidden during a `foreach`; |
||||
non-structural updates of existing elements can still be done with assignment |
||||
operators. |
||||
- Fixed native typed object properties cannot be freely `unset()` with PHP's |
||||
standard uninitialized-property semantics. |
||||
- Calling `unset()` on a native-typed variable does not delete the variable as |
||||
it would in standard PHP. |
||||
|
||||
## Runtime dynamic capabilities |
||||
|
||||
- Ordinary Zend objects and dynamic class expressions support runtime `::class` |
||||
and class-constant lookup. Native Classes still require the relevant class |
||||
target to be known at compile time. |
||||
- `static::class` is not supported in positions that require a compile-time |
||||
constant class name. |
||||
- `__CLASS__` may only be used within a `class` definition (PHP allows |
||||
it elsewhere and returns an empty string). |
||||
- `__TRAIT__` may only be used within a `trait` definition (PHP allows |
||||
it elsewhere and returns an empty string). |
||||
- Dynamic property chains, dynamic class names, dynamic function names, and |
||||
dynamic callbacks all go through the Zend runtime fallback and are not |
||||
guaranteed to be natively optimized; reference parameters of dynamic calls |
||||
still require an explicit `refval()` or `toRef()`. |
||||
- `Closure::bind()`, `Closure::bindTo()`, and `Closure::call()` are not |
||||
supported. A closure cannot be rebound to an object or class scope in AOT |
||||
code. |
||||
- All source files must be encoded as `UTF-8`. |
||||
|
||||
## Generators |
||||
|
||||
TypePHP generators use `FiberGenerator` rather than Zend `Generator`. Code must |
||||
therefore not rely on `instanceof Generator`, `ReflectionGenerator`, Zend |
||||
Generator's internal object layout, or identical exception traces. Generators |
||||
do not support returning by reference, by-reference yield, by-reference |
||||
`foreach`, by-reference parameters, or variadic parameters. Fiber and Generator |
||||
are not currently supported by the WASI target. See |
||||
[Generators](YIELD_GENERATOR.md) for the complete boundary. |
||||
@ -0,0 +1,116 @@ |
||||
# Interface Property Hooks Implementation Plan |
||||
|
||||
This document records the design and implementation plan for TP-AOT-010. The goal is to support the PHP 8.4 Interface Property Hook contract while preserving the zero-cost abstraction of TypePHP Native calls, and to give the PHP 8.4 ZendVM complete metadata for Reflection, dynamic class linking, and inheritance checks. |
||||
|
||||
## Current status (2026-08-14) |
||||
|
||||
The first stage has landed: the Interface contract model, AOT implementation checks, get/set direction variance, PHPX abstract Hook metadata, Reflection, dynamic PHP implementation classes, and regression tests are all wired up. Explicit setter parameter types are still rejected at compile time per the convention below; they will be opened up once the independent write-type model is completed. |
||||
|
||||
## 1. Design conclusion |
||||
|
||||
A Hooked Property in an Interface only represents a property contract: |
||||
|
||||
```php |
||||
interface Named |
||||
{ |
||||
public string $name { get; set; } |
||||
} |
||||
``` |
||||
|
||||
- The Interface holds no property slot, generates no getter/setter implementation, and produces no contract check at access time. |
||||
- TypePHP verifies at compile time whether known AOT classes satisfy the property's visibility, type, and `get`/`set` capabilities. |
||||
- The PHP 8.4 target registers native Zend Hook metadata in MINIT so Reflection and dynamic PHP classes obtain the same contract. |
||||
- The minimum version of TypePHP, PHPX, and the final target runtime is PHP 8.4; no downgrade path is provided for older versions. |
||||
|
||||
## 2. Syntax and diagnostics |
||||
|
||||
Three kinds of contracts are supported: |
||||
|
||||
```php |
||||
public string $readable { get; } |
||||
public string $writable { set; } |
||||
public string $readWrite { get; set; } |
||||
``` |
||||
|
||||
An Interface Property Hook must be `public`, non-`static`, have no default value, and its Hooks must not contain function bodies. Ordinary Interface properties, `private`/`protected`, `readonly`, duplicate or unknown Hooks, and Hooks with implementation bodies all throw a FatalError at TypePHP compile time. Error messages should match PHP 8.4 as closely as possible. |
||||
|
||||
The first stage only accepts the implicit setter parameter: |
||||
|
||||
```php |
||||
public string $name { set; } |
||||
``` |
||||
|
||||
PHP 8.4 also allows explicit, contravariant setter parameters such as `set(string|Stringable $value)`. That syntax requires the compile-time contract model and the Zend Hook `arg_info` to simultaneously store a write type independent of the property read type; until this part is complete, TypePHP reports a clear compile-time error and does not generate potentially incorrect runtime metadata. |
||||
|
||||
## 3. Compiler model |
||||
|
||||
An Interface Property Hook must not be disguised as an ordinary property or as an ordinary method after lowering. A separate contract model is established, storing at least: |
||||
|
||||
- the property name and declaration node; |
||||
- the resolved TypePHP type and class type; |
||||
- whether `get` is required; |
||||
- whether `set` is required; |
||||
- visibility and other flags used for diagnostics. |
||||
|
||||
The contract is stored in `InterfaceDef`. The AST/preprocessing stage only collects and validates declarations; it does not allocate property slots for Interfaces, does not run the `PropertyHookLowering` used by concrete classes, and does not generate hidden methods. |
||||
|
||||
Contract linking is performed after all types finish preprocessing: parent Interface contracts are expanded, then the properties provided by the implementing class itself or its parent are checked. An ordinary public backed property satisfies both the read and write contracts; a Hooked Property is judged by its actual Hook capabilities. get-only types are covariant in the read direction, set-only types are contravariant in the write direction, and types containing both get and set remain invariant. |
||||
|
||||
## 4. PHPX and Zend metadata |
||||
|
||||
The existing `typephp_register_property_hooks()` is for concrete classes with real AOT getters/setters and cannot be reused for abstract Interface Hooks. |
||||
|
||||
PHPX adds a separate helper: |
||||
|
||||
```cpp |
||||
typephp_register_abstract_property_hooks( |
||||
zend_class_entry *interface_ce, |
||||
zend_property_info *property_info, |
||||
bool readable, |
||||
bool writable |
||||
); |
||||
``` |
||||
|
||||
TypePHP/PHPX already uniformly require PHP 8.4+, so this helper directly accesses the PHP 8.4 ABI and is responsible for: |
||||
|
||||
- persistently allocating `zend_property_info::hooks`; |
||||
- creating abstract `get`/`set` `zend_internal_function` metadata without handlers; |
||||
- setting `ZEND_ACC_PUBLIC | ZEND_ACC_ABSTRACT`, the correct parameter/return types, and `common.prop_info`; |
||||
- updating `num_hooked_props` so Zend inheritance and Reflection recognize the contract; |
||||
- ensuring all strings, Hook tables, and function descriptors have MINIT-level persistent lifetimes. |
||||
|
||||
The generated code first registers the Interface, then declares the property with `IS_UNDEF` and `ZEND_ACC_PUBLIC | ZEND_ACC_ABSTRACT | ZEND_ACC_VIRTUAL` and mounts the abstract Hooks, and finally registers and links the implementing classes. |
||||
|
||||
## 5. PHP version boundary |
||||
|
||||
TypePHP distinguishes the source language version from the linked runtime: |
||||
|
||||
- `--php-version` allows only `8.4` or `8.5` and is used to parse syntax and handle project conditions; |
||||
- PHPX headers, `libphp`, and the final runtime must be PHP 8.4 or higher; |
||||
- `--php-version` and `libphp.so` are not required to have exactly matching minor versions — for example, when using the 8.5 syntax mode and linking PHP 8.4, whether the final build succeeds is still determined by the Zend APIs actually used; |
||||
- environments below PHP 8.4 are rejected directly at the TypePHP/PHPX build entry point. |
||||
|
||||
## 6. TDD coverage |
||||
|
||||
Add failing tests before implementation, covering: |
||||
|
||||
1. get-only, set-only, and get/set Interface contracts; |
||||
2. ordinary backed properties, Hooked Properties, and inherited properties satisfying the contract; |
||||
3. compile errors for missing properties, missing get/set, non-public, and incompatible types; |
||||
4. Interface inheritance, merging of multiple contracts, and conflicts; |
||||
5. Reflection abstract, virtual, hasHook/getHook metadata; |
||||
6. success and failure linking of PHP 8.4 dynamic PHP classes; |
||||
7. consistent O0/O3 results, with Interfaces generating no property slots or Native Hook implementations; |
||||
8. lifetime and ABI regression of the PHPX helper under NTS/ZTS and PHP 8.4/8.5. |
||||
|
||||
## 7. Implementation order |
||||
|
||||
1. Add TP-AOT-010 normal-scenario and syntax-error PHPT and confirm the current failures. |
||||
2. Add the Interface Property Contract model and preprocessing collection logic. |
||||
3. Implement Interface inheritance and compile-time contract checking for implementing classes. |
||||
4. Add the abstract Hook metadata helper in PHPX. |
||||
5. Modify stub generation and class registration order to wire into the PHP 8.4 Zend metadata. |
||||
6. Add Reflection, dynamic class linking, target version, and generated-code tests. |
||||
7. Run the Interface, Property Hook, Reflection, and full compiler regression. |
||||
|
||||
After completion, runtime property access still goes directly into the implementing class's ordinary properties or Native Hooks; the Interface contract itself exists only in the compile-time model and MINIT metadata, and does not enter the request hot path. |
||||
@ -0,0 +1,67 @@ |
||||
# Automatically building libphp.so |
||||
|
||||
TypePHP's executable and shared-library modes require the `libphp.so` provided by the PHP Embed SAPI. Many Linux distributions' PHP packages only include CLI or FPM, so the Composer-installed `tpc.php` asks whether to automatically build a private PHP when `libphp.so` is not found. |
||||
|
||||
The installer targets `tpc.php` launched by the PHP interpreter. The bootstrap binary `tpc` must have `libphp.so` and `libphpx.so` loaded by the system dynamic linker before entering `main()`, so it cannot install missing libraries by itself. |
||||
|
||||
This feature is only enabled on Linux and in interactive terminals. Extension mode (`-m ext`) does not need `libphp.so` and does not trigger the installer; non-interactive environments such as CI also do not automatically download or install packages, or execute `sudo`. |
||||
|
||||
## Usage flow |
||||
|
||||
Simply run the normal compile command: |
||||
|
||||
```bash |
||||
vendor/bin/tpc.php project.yml |
||||
``` |
||||
|
||||
When `libphp.so` is missing, the installer asks in sequence: |
||||
|
||||
1. whether to automatically build the PHP Embed library; |
||||
2. the PHP version, defaulting to exactly the `PHP_VERSION` of the currently running `tpc.php`, with the option to manually enter another PHP 8.4.x/8.5.x stable version; |
||||
3. the install directory, defaulting to `~/.typephp`; |
||||
4. whether to install missing development packages via the detected `apt-get`, `dnf`, or `yum`. |
||||
|
||||
The installer reads the current `php-config --configure-options`, keeps the current PHP's extension configuration, replaces the install path, and adds `--enable-embed=shared`. PHP source is downloaded only from PHP.net, and verified using the SHA-256 from the official release information. |
||||
|
||||
After compilation, the main files are as follows: |
||||
|
||||
```text |
||||
~/.typephp/bin/php |
||||
~/.typephp/bin/php-config |
||||
~/.typephp/lib/libphp.so |
||||
~/.typephp/lib/php.ini |
||||
~/.typephp/lib/loaded-extensions.txt |
||||
``` |
||||
|
||||
The current main ini file and the configuration in the scan directory are merged. When the same PHP major/minor version is used, the shared extensions loaded in the current ini are copied to the new extension directory; across major/minor versions, binary extensions are not copied, and unusable extension configuration is commented out to avoid a generated PHP that cannot start. |
||||
|
||||
After a successful installation, the current `tpc.php` process automatically uses the new directory as `PHP_HOME` and continues the original compile task. It can also be specified explicitly later: |
||||
|
||||
```bash |
||||
export PHP_HOME="$HOME/.typephp" |
||||
vendor/bin/tpc.php project.yml |
||||
``` |
||||
|
||||
When the same directory and version are chosen again, the installer asks whether to directly reuse the existing `libphp.so` and does not repeat the full build. |
||||
|
||||
## Non-interactive environments |
||||
|
||||
The installer does not automatically confirm privileged operations in CI. Prepare `libphp.so` in advance, then set: |
||||
|
||||
```bash |
||||
PHP_HOME=/path/to/php vendor/bin/tpc.php project.yml |
||||
``` |
||||
|
||||
## Automatically building libphpx.so |
||||
|
||||
After the PHP Embed check completes on Linux, the build pipeline also checks `lib/libphpx.so` in the PHPX root directory. The PHPX root directory is resolved in the following order: |
||||
|
||||
1. `PHPX_HOME`; |
||||
2. the `swoole/phpx` install path in Composer `InstalledVersions`; |
||||
3. `vendor/swoole/phpx` within the TypePHP source repository. |
||||
|
||||
When the shared library is missing, an interactive terminal asks whether to build. PHPX itself does not depend on `libphp.so`; it depends on PHP headers and `php-config`. `LibPhpxInstaller` uses the currently selected PHP prefix, sets `PHP_HOME`, puts that prefix's `bin` at the front of `PATH`, and passes `-Dphp_dir=<PHP prefix>` to the PHPX CMake, ensuring PHPX matches the PHP ABI used by the project runtime. |
||||
|
||||
Toolchain detection runs after the local library checks. After the user confirms the automatic build, the installer checks and can install GCC/G++, make, CMake, pkg-config, and the dependencies required by PHP configure via `apt-get`, `dnf`, or `yum`; therefore a Composer environment does not need a full pre-installed C/C++ toolchain. |
||||
|
||||
The build always uses Release, disables PHPX tests, and builds only the `phpx` target; parallelism is capped at 8. The output must be `<PHPX>/lib/libphpx.so`, otherwise the installer reports an error. Non-interactive environments only report the missing library and do not run CMake. |
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,136 @@ |
||||
# Native Class Implementation Acceptance Matrix |
||||
|
||||
> Audit date: 2026-08-17 |
||||
> This document records the requirements, implementation entry points, and direct verification evidence for the `#[Native]` object model. It is the implementation-acceptance attachment for |
||||
> [NATIVE_CLASS_OBJECT.md](NATIVE_CLASS_OBJECT.md) and does not replace the semantic design document. |
||||
|
||||
## 1. Acceptance principles |
||||
|
||||
Every capability must simultaneously have: |
||||
|
||||
1. a clear language boundary; |
||||
2. a locatable compiler or PHPX implementation; |
||||
3. direct evidence in positive PHPT, negative PHPUnit, or PHPX C++ unit tests. |
||||
|
||||
Code alone, documentation alone, or "no failure currently observed" does not count as complete. Native Objects have no Zend representation, so any cross-boundary behavior that cannot be statically proven safe must be rejected before generating C++. |
||||
|
||||
## 2. Object model and code generation |
||||
|
||||
| Requirement | Implementation evidence | Test evidence | Conclusion | |
||||
|---|---|---|---| |
||||
| `#[Native]` used only on named classes | `NativeClassAttributeLowering`, `NativeClassSupportTrait` | `testRejectsNativeAttributeOnInterface/Trait/Enum/AnonymousClass` | Verified | |
||||
| No Zend class/object handlers registered | Native struct, descriptor, and free-function generation path | `clone-and-zend-invisible.phpt`, Reflection negative tests | Verified | |
||||
| Methods keep the `php_*` free-function ABI | Native method/virtual thunk generation path | `basic.phpt`, `chained-call.phpt` | Verified | |
||||
| Statically resolvable `new NativeClass()` uses the Native Heap | `CompilerBase::parseNew()`, `php::nativeConstruct()` | `basic.phpt`, `construction-gc-roots.phpt` | Verified | |
||||
| `new (expression)()` stays ordinary PHP dynamic instantiation | `parseNew()` enters the Native branch only for `Node\\Name` | `testLeavesDynamicClassExpressionsToTheOrdinaryPhpPath` | Verified | |
||||
| A Native object itself cannot serve as a dynamic class target | `assertNotNativeObjectDynamicClassTarget()` | dynamic new/static call/class constant negative tests | Verified | |
||||
| All unsupported usages terminate at compile time | Native boundary checks, type compatibility checks | 131 `NativeClassValidationTest` items | Verified | |
||||
|
||||
## 3. Properties and fixed layout |
||||
|
||||
| Requirement | Implementation evidence | Test evidence | Conclusion | |
||||
|---|---|---|---| |
||||
| All properties must declare types | Native field validation | `testRejectsUntypedProperty` | Verified | |
||||
| bool/int/float use fixed value fields | Native field C++ type mapping | `basic.phpt`, `numeric-properties.phpt` | Verified | |
||||
| string/array/object/typed object/Stream/mixed usable as fields | Native PHPX field mapping, write checks | `phpx-properties.phpt`, `stream-property.phpt`, `composite-property-types.phpt` | Verified | |
||||
| BigInt/BigFloat/Decimal usable as fields | high-precision field mapping and trace/destroy | `high-precision-properties.phpt` | Verified | |
||||
| Native type fields hold raw pointers and can form cyclic types | struct forward declaration, descriptor trace | `mutual-reference-types.phpt`, `gc-cycle.phpt` | Verified | |
||||
| Fields without explicit initialization use deterministic zero values | Native field initializer | `zero-values.phpt` | Verified | |
||||
| Property writes keep the declared type | Native property assignment validation | composite, stream, and multiple negative PHPUnit | Verified | |
||||
| Only `any` properties may take PHP references | Native property reference lowering | `any-property-reference.phpt` and mixed/fixed property negative tests | Verified | |
||||
| `unset()` not supported on Native properties | property unset validator | `testRejectsUnsetOnNativeObjectProperties` | Verified | |
||||
| readonly properties not supported | Native declaration validator | `testRejectsReadonlyPropertyUntilNativeWriteStateIsImplemented` | Verified | |
||||
| Box/Std Container cannot be embedded in fields | Native field validator | Box/Std Container property negative tests | Verified | |
||||
|
||||
## 4. Identity, nullability, and call ABI |
||||
|
||||
| Requirement | Implementation evidence | Test evidence | Conclusion | |
||||
|---|---|---|---| |
||||
| `$a = $b` only copies the pointer and shares object identity | Native pointer local representation | `parameter-semantics.phpt` | Verified | |
||||
| Native parameters and returns must explicitly declare concrete classes | call argument/return boundary validation | untyped/mixed/interface parameter and return negative tests | Verified | |
||||
| Ordinary Native parameters are non-null; only `?Class` may be null | function entry/return checks | `non-null-parameter.phpt`, `nullable-signatures.phpt`, `return-nullability.phpt` | Verified | |
||||
| `&` forbidden on Native parameters, returns, and variables | reference boundary validation | reference parameter/return/assignment/function/method negative tests | Verified | |
||||
| Native variadic, union/intersection signatures not supported | signature validation | variadic/union/null-union negative tests | Verified | |
||||
| `unset($object)`/`$object = null` only clear the current pointer slot | Native root slot lowering | `unset-alias.phpt` | Verified | |
||||
| `===`/`!==` and `match` use pointer identity | Native identity lowering | `strict-identity.phpt`, `match-identity.phpt` | Verified | |
||||
| ternary/match/coalesce choose the nearest common Native base class for sibling subclasses | `getCommonNativeObjectClass()`, selection pointer cast | `value-selection.phpt`, cross-file global discovery tests | Verified | |
||||
| Conditional expressions check for non-null pointer without calling `toBool()` | Native condition lowering | `conditions.phpt` | Verified | |
||||
| Loose comparison, arithmetic, bitwise, increment/decrement, compound writes, and switch forbidden | operator validators | corresponding PHPUnit negative tests | Verified | |
||||
| `isset`/`empty`/`is_null`/nullsafe keep the typed pointer | Native selection/nullsafe lowering | `isset-empty.phpt`, `is-null.phpt`, `nullsafe.phpt` | Verified | |
||||
| Call arguments strictly evaluated left-to-right and precisely rooted at safe points | Native call argument materialization | `call-argument-roots.phpt`, `constructor-argument-roots.phpt` | Verified | |
||||
|
||||
## 5. Class language capabilities |
||||
|
||||
| Requirement | Implementation evidence | Test evidence | Conclusion | |
||||
|---|---|---|---| |
||||
| Single inheritance, abstract, and limited virtual dispatch | Native C++ inheritance/virtual slot adapters | `abstract-method.phpt`, `polymorphic-clone.phpt`, `virtual-signature-variance.phpt` | Verified | |
||||
| public/private/protected checked at compile time | Native member resolution | `method-visibility.phpt` and inaccessible method/constant negative tests | Verified | |
||||
| Traits compiled as ordinary Native members after injection | existing Trait AST injection + Native member generation | `trait-inheritance-interface.phpt` | Verified | |
||||
| Interfaces are compile-time contracts only and cannot become value representations | interface contract validator | `internal-interface.phpt`, `interface-property-hooks.phpt`, and interface escape negative tests | Verified | |
||||
| Compile-time resolvable `instanceof` folds | Native instanceof lowering | `instanceof.phpt`, dynamic instanceof negative tests | Verified | |
||||
| Getter/Setter annotations generate direct calls | annotation lowering + Native method path | `generators.phpt` | Verified | |
||||
| Property Hooks support only direct get/set | Native hook lowering | `property-hooks.phpt`, `property-hook-native-object.phpt`, and indirect operation negative tests | Verified | |
||||
| `clone` preserves dynamic subclass, PHPX COW, and shallow object semantics | Native clone descriptor/thunk, `php::nativeClone()` | clone PHPT series, `clone-phpx-fields.phpt` | Verified | |
||||
| `__construct` called only by `new` | Native construction path, explicit-call checks | construction PHPT series, explicit constructor negative tests | Verified | |
||||
| `__destruct` executed at most once by GC, derived-to-base along the inheritance chain | Native finalizer chain | destructor/finalizer/lifecycle PHPT series | Verified | |
||||
| `__invoke` and `__toString` use a deterministic Native Call | Native magic method allow-list | `magic-methods.phpt` | Verified | |
||||
| Dynamic magic methods, variable property/method names not supported | Native magic/dynamic access deny-list | dynamic magic, variable method/property negative tests | Verified | |
||||
| `toArray/toString/toInt/toFloat/toBool/toObject` require a real method, zero parameters, and an exact return type | Native keyword method resolution | `keyword-conversions.phpt`, `testNativeObjectToObjectKeywordUsesDeclaredNativeMethod`, and signature negative tests | Verified | |
||||
| `count($obj)` specialized only when implementing Countable | Native count optimizer | `keyword-conversions.phpt`, count-without-countable negative tests | Verified | |
||||
| `ArrayAccess` direct syntax maps to Native `offset*()` methods | Native array access lowering | `array-access.phpt` | Verified | |
||||
| Native `ArrayAccess` forbids indirect modification and references | writable-chain/reference validators | ArrayAccess compound/increment/nested/property/reference/coalesce negative tests | Verified | |
||||
| Native `Iterator` foreach maps to protocol methods, preserving PHP call order | Native foreach lowering | `iterator.phpt` | Verified | |
||||
| `IteratorAggregate` routes Native Iterator vs PHP Traversable | aggregate return-type lowering | `iterator.phpt` | Verified | |
||||
| Native foreach does not enumerate properties and forbids by-reference traversal | interface/reference validators | foreach negative PHPUnit | Verified | |
||||
|
||||
## 6. GC and lifetime |
||||
|
||||
| Requirement | Implementation evidence | Test evidence | Conclusion | |
||||
|---|---|---|---| |
||||
| Wren-style precise, non-moving, STW mark-sweep | `phpx/thirdparty/wren-gc`, `native_gc.cc` | PHPX `wren_gc.*` | Verified | |
||||
| Raw pointer writes have no RC, no write barrier | Native pointer field/local codegen | generated C++ review, Native PHPT | Verified | |
||||
| 16 MiB initial threshold, 1 MiB lower bound, 50% headroom | Wren GC configuration | `wren_gc.uses_stable_native_heap_defaults` | Verified | |
||||
| Precise root frames keep the object graph alive | `NativeRootFrame`, generated root slots | PHPX root tests, `gc-cycle.phpt` | Verified | |
||||
| Fiber non-LIFO lifetime safety | root frame registry | `fiber-lifetime.phpt`, `fiber-shutdown.phpt`, PHPX Fiber root tests | Verified | |
||||
| global/static request roots are thread-local under ZTS | generated globals/root registration | `global-and-static.phpt` under ZTS, PHPX request root tests | Verified | |
||||
| RSHUTDOWN clears roots and destroys the heap | `nativeGcRequestShutdown()` | PHPX shutdown tests | Verified | |
||||
| A finalizer can resurrect once and is not re-executed afterward | Wren/Native finalization state | `gc-cycle.phpt`, PHPX resurrection tests | Verified | |
||||
| Allocation, exceptions, and Zend state safe in finalizers | finalizer queue/exception cleanup | finalizer/lifecycle PHPT, PHPX finalizer tests | Verified | |
||||
| Construction or clone failure leaves no dangling object, and escaped objects remain valid | `nativeConstruct()`, `nativeClone()` failure paths | `failed-lifecycle-escape.phpt`, `failed-clone-finalizer.phpt` | Verified | |
||||
|
||||
## 7. ZendVM boundary and containers |
||||
|
||||
| Requirement | Implementation evidence | Test evidence | Conclusion | |
||||
|---|---|---|---| |
||||
| Native Objects cannot enter PHP array/object property/mixed | escape and boundary validators | corresponding PHPUnit negative tests | Verified | |
||||
| Cannot be passed to PHP/ZendVM dynamic functions, Closures, or constructors | call boundary validator | dynamic call, Closure, Zend constructor negative tests | Verified | |
||||
| Reflection/WeakReference/serialize/json_encode not supported | facility-specific diagnostics | corresponding PHPUnit negative tests | Verified | |
||||
| Generators cannot hold, receive, or yield Native pointers | generator boundary validator | generator series negative tests | Verified | |
||||
| Native locals of ordinary functions across Fiber suspend have precise roots | root frame lifecycle | Fiber PHPT | Verified | |
||||
| Local Std Containers can hold concrete Native pointers | Std Container Native value mapping/root frame | `std-containers.phpt` | Verified | |
||||
| Native Std Containers cannot escape as Zend values, static/global, or closure capture | container escape validation | Std Container series negative PHPUnit | Verified | |
||||
| `include`/`eval` do not expose Native locals to the Zend symbol table | include scope filtering | `include-native-scope.phpt` | Verified | |
||||
|
||||
## 8. Project-level analysis |
||||
|
||||
| Requirement | Implementation evidence | Test evidence | Conclusion | |
||||
|---|---|---|---| |
||||
| Native class forward declarations do not depend on file order | declaration discovery pre-pass | `testDiscoversNativeTypesBeforeCrossFileSignaturePreprocessing` | Verified | |
||||
| Global Native slot ABI fixed before any C++ file is generated | `NativeGlobalDiscovery`, `NativeGlobalTypeResolver` | `testDiscoversNativeGlobalSlotBeforeEarlierReaderIsConverted`, actual dual-file build | Verified | |
||||
| `global $slot` and statically resolvable `$GLOBALS[...]` use the same Native root slot | literal/constant global slot lowering, request root registration | `global-and-static.phpt`, cross-file Closure/constant `$GLOBALS` fixture | Verified | |
||||
| Dynamic `$GLOBALS[$key]` must not carry Native Objects | dynamic Zend boundary validation | `testRejectsNativeObjectStoredThroughDynamicGlobalsKey` | Verified | |
||||
| Global slot fixes the first Native type and allows only subclasses or null | global registration/type validation | `global-and-static.phpt`, global type change negative tests | Verified | |
||||
| Projects without Native Classes skip the Native global pre-pass | `discoverNativeGlobalObjects()` fast return | source review, full PHPUnit | Verified | |
||||
|
||||
## 9. Current verification commands |
||||
|
||||
```bash |
||||
./run-tests.php -j4 --compiler ./tpc tests/compiler/native-class/ |
||||
vendor/bin/phpunit phpunit/src/NativeClass/NativeClassValidationTest.php |
||||
/home/swoole/workspace/aot/phpx/build/bin/phpx-tests \ |
||||
--gtest_filter='wren_gc.*:native_gc.*' |
||||
``` |
||||
|
||||
The results of this Iterator-focused run were: `iterator.phpt` 1/1, Native Class PHPUnit 136/136, |
||||
ordinary foreach regression 14/14. The Native Class PHPT directory currently has 71 items; per the current task agreement, the full |
||||
tests for that directory and the compiler PHPT suite were not re-run this time, and are left for the next unified regression round. |
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,212 @@ |
||||
# Zend Object Creation and Property Default Value Initialization |
||||
|
||||
This document records the initialization responsibilities of the Zend Classes generated by TypePHP during MINIT and object creation, focusing on when a custom `create_object` is required, what behavior is allowed within it, and the performance boundary on the object-creation hot path. |
||||
|
||||
This document only discusses ordinary TypePHP Classes registered with ZendVM. `#[Native]` Classes use the Native Heap and GC and do not follow the flow described here. |
||||
|
||||
## 1. The two initialization stages must be kept separate |
||||
|
||||
Property initialization of a TypePHP Class is split into two stages: |
||||
|
||||
1. `gen_stub.php` generates `register_class_*()` during MINIT, establishing the `zend_class_entry`, property metadata, and the default property table; |
||||
2. only values that cannot be accurately expressed by the default property table are supplemented by a custom `create_object` each time an object is created. |
||||
|
||||
These two stages must not perform the same property assignment twice. Values already written by `register_class_*()` are copied to the new object by Zend's `object_properties_init()`; calling `zend_update_property()` again has no semantic value and additionally enters the property-name lookup, type check, handler dispatch, and reference-counting paths. |
||||
|
||||
## 2. Default values handled by gen_stub.php |
||||
|
||||
The following values can be accurately written into the Zend Class default property table: |
||||
|
||||
| Source default value | Registration-stage representation | Must be written again in `create_object` | |
||||
|---|---|---| |
||||
| `null` | `ZVAL_NULL` | No | |
||||
| `bool` | `ZVAL_TRUE/FALSE` | No | |
||||
| `int` | `ZVAL_LONG` | No | |
||||
| `float` | `ZVAL_DOUBLE` | No | |
||||
| `string` | persistent `zend_string` | No | |
||||
| scalar constant expression | the scalar zval evaluated at compile time | No | |
||||
| `[]` | `ZVAL_EMPTY_ARRAY` | No | |
||||
| TypePHP typed property without an explicit default | the zero value, empty string, empty array, `null`, or `UNDEF` defined by TypePHP | No | |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
class Value |
||||
{ |
||||
private const BASE = 20; |
||||
|
||||
public int $id = self::BASE + 3; |
||||
public string $name = 'type' . 'php'; |
||||
public array $items = []; |
||||
} |
||||
``` |
||||
|
||||
As long as the expressions can be safely evaluated at compile time, the three properties above should rely entirely on the Zend Class default property table. When creating `Value`, `zend_update_property()` must not be called again. |
||||
|
||||
## 3. When default values need runtime supplementation |
||||
|
||||
The current `gen_stub.php` cannot accurately represent the following values in the default property table. |
||||
|
||||
### 3.1 Non-empty arrays |
||||
|
||||
Non-empty array defaults currently use `ZVAL_EMPTY_ARRAY` as a placeholder value in the registration function. Each object must construct an independent, semantically correct array value: |
||||
|
||||
```php |
||||
class Request |
||||
{ |
||||
public array $options = ['timeout' => 10]; |
||||
} |
||||
``` |
||||
|
||||
Therefore `Request::$options` needs to be supplemented in `create_object`. Multiple objects still follow PHP array copy-on-write semantics; modifying one object's array must not affect other objects. |
||||
|
||||
Array constants follow the same rule. If the compiler can only determine that it is an array but cannot prove it is empty, it conservatively keeps the runtime initialization. |
||||
|
||||
### 3.2 Enum case |
||||
|
||||
An enum case is an object, not a scalar constant: |
||||
|
||||
```php |
||||
enum State |
||||
{ |
||||
case Ready; |
||||
} |
||||
|
||||
class Task |
||||
{ |
||||
public State $state = State::Ready; |
||||
} |
||||
``` |
||||
|
||||
The class registration code currently can only generate a placeholder value first; `create_object` then obtains the real enum case object and writes it into the property. Therefore "only non-empty arrays need a custom `create_object`" is not correct — enum case is a clear second category of counterexample. |
||||
|
||||
### 3.3 Constant expressions that cannot be safely resolved |
||||
|
||||
If the preprocessing stage cannot prove that a default value can be accurately expressed by the Zend default property table, the compiler must conservatively keep the runtime initialization. Optimization can only remove work that is proven redundant; it must not guess the runtime type based on the expression's shape. |
||||
|
||||
## 4. handlers and parent allocator |
||||
|
||||
### 4.1 Property Hooks and asymmetric set visibility do not trigger on their own |
||||
|
||||
PHP 8.4 Property Hooks, `private(set)`, and `protected(set)` install TypePHP custom object handlers, but this by itself does not require overriding `create_object`. Zend 8.4's `object_properties_init()` directly copies the class default table and does not call read/write handlers; ordinary `php::stdCreateObject()` already sets the final handlers correctly. |
||||
|
||||
A custom creation flow is required only when the class also has runtime defaults such as non-empty arrays or enum cases. Supplemental initialization must bypass setters; even using `zend_std_write_property()`, PHP 8.4 would call the setter based on the Hook metadata. The current generated code therefore uses the property offset known at compile time to directly update the backing slot via PHPX `Object::attr(offset)`. |
||||
|
||||
### 4.2 Parent custom object allocator |
||||
|
||||
If the parent class comes from a built-in PHP extension, or an ancestor class has a custom object storage layout, the child class cannot bypass the parent's allocator. When the current class genuinely needs a custom creation flow due to runtime defaults, it must first call the saved parent `create_object`, then supplement the current class's values. |
||||
|
||||
When the TypePHP parent has already installed a custom allocator, ordinary child classes usually inherit it directly. A new delegation layer is generated only when the child class itself also needs supplemental initialization. |
||||
|
||||
## 5. Execution flow of a custom create_object |
||||
|
||||
The generated code performs the following steps through `typephp_create_object_with_defaults()`: |
||||
|
||||
1. save the class's final `default_object_handlers`; |
||||
2. if the parent object layout must be respected, call the saved parent allocator; otherwise run `zend_objects_new()` and `object_properties_init()`; |
||||
3. temporarily switch the new object to Zend standard object handlers to keep the exception path and other object operations in a controlled state; |
||||
4. initialize only the properties marked `requiresRuntimeDefaultInit`, writing directly to backing slots via cached declared-property offsets; |
||||
5. check for Zend exceptions after each write; |
||||
6. restore the final handlers whether returning normally or after a C++ exception; |
||||
7. return the fully initialized `zend_object *`. |
||||
|
||||
The initializer is a template parameter and a compile-time lambda; it does not use `std::function` and does not dynamically allocate memory for the lambda. `delegate_to_base` is a call-site-determined boolean that can usually be folded by the C++ compiler in optimized builds. |
||||
|
||||
The following behavior does not belong to `create_object`: |
||||
|
||||
- the function body of PHP `__construct()`; |
||||
- static property default-value initialization; that happens in `module_init()`; |
||||
- scalar, `null`, and empty-array assignments already expressed by the default property table; |
||||
- reapplying defaults after clone; clone should copy the source object's current state, not recreate the default state. |
||||
|
||||
## 6. Main performance issues already fixed |
||||
|
||||
The old generation logic installed a custom `create_object` whenever any explicit non-static default existed in the class, and re-updated all default properties on every object creation. This produced two layers of duplicate cost: |
||||
|
||||
1. an ordinary class containing only `public int $value = 0` also bypassed the standard fast creation path; |
||||
2. a class containing even one non-empty array caused all other scalar properties to be re-updated one by one. |
||||
|
||||
The current rules have been adjusted to: |
||||
|
||||
- only properties that truly need runtime supplementation trigger `requireCtor`; |
||||
- properties already accurately registered by `gen_stub.php` do not appear in the runtime initialization block; |
||||
- classes with only Hooks/asymmetric visibility and no runtime defaults no longer generate an empty custom allocator; |
||||
- when Hooks and runtime defaults coexist, a fixed property offset updates the backing slot without calling the setter. |
||||
|
||||
In a micro benchmark, `new Foo()` containing only scalar properties dropped from about `1.8s` to about `0.78s`, close to the approximately `0.83s` of ZendPHP in the same environment after subtracting the empty loop. This number is only used to record the magnitude of the optimization, not a cross-machine performance promise. |
||||
|
||||
## 7. Implemented optimizations, remaining costs, and future directions |
||||
|
||||
### 7.1 Non-empty arrays use a request-level template and copy-on-write |
||||
|
||||
Non-empty arrays cannot be placed in an internal class's default property table, but that does not mean the array must be rebuilt for every object. The current generator already uses request-level default-value templates: |
||||
|
||||
1. each class containing runtime array defaults owns a set of `THREAD_LOCAL php::Var` templates and an initialization state; NTS builds introduce no locking; |
||||
2. the first time an object of the class is created, all its templates are lazily built via `UNEXPECTED(!initialized)`; |
||||
3. templates are committed and the initialization flag is set only after all templates are successfully built in local temporaries; construction exceptions do not publish a half-initialized state; |
||||
4. template initialization happens before object allocation, so a failure leaves no unreturned object; |
||||
5. subsequent object creations just copy the template zval into the target backing slot, i.e. increment the array reference count once; |
||||
6. the first time an object modifies that property, Zend/PHPX's `SEPARATE_ARRAY` performs copy-on-write; |
||||
7. `module_clean()` releases the templates and resets the initialization state; the HashTable allocated by the request allocator does not survive RSHUTDOWN. |
||||
|
||||
Take the following default value as an example: |
||||
|
||||
```php |
||||
class Request |
||||
{ |
||||
public array $options = [ |
||||
'timeout' => 10, |
||||
'headers' => ['Accept' => 'application/json'], |
||||
]; |
||||
} |
||||
``` |
||||
|
||||
If ten thousand objects are created but `$options` is not modified, the array and nested arrays are built only once; each object only holds the shared zval. If one object executes `$request->options['timeout'] = 30`, only that object is separated at write time, while the other objects and the template remain unchanged. Nested arrays also continue to use Zend's existing per-level copy-on-write rules. |
||||
|
||||
PHP property default arrays cannot contain references, and the objects allowed in constant expressions are mainly immutable enum cases, so sharing the template conforms to default-property semantics. PHPT already covers top-level writes, nested writes, `unset`, reference writes, and dynamic object writes, confirming that these paths all separate correctly. |
||||
|
||||
A persistent array cannot simply be constructed in MINIT and passed to `zend_declare_typed_property()`. TypePHP registers `ZEND_INTERNAL_CLASS`, and Zend 8.4 explicitly forbids internal properties from using refcounted default zvals; the internal-class fast path of `_object_properties_init()` also does not increment the reference count of defaults. Non-empty arrays and enum objects are both refcounted values. |
||||
|
||||
Therefore, without changing the foundational design of "TypePHP Classes are registered as internal classes" and without modifying the Zend ABI, non-empty arrays still do not enter the class default table; the out-of-table request-level template reduces array construction cost from "once per object" to "once per request per default value". Objects that do not modify the default array only bear the zval copy and reference-counting cost; only objects that actually modify it bear the array-separation cost. |
||||
|
||||
Templates are initialized lazily per class rather than unconditionally building all templates in RINIT: in large projects, many classes are never instantiated within a single request. Each object only adds one highly predictable initialization-state branch; after the first time, the branch stably evaluates to false. |
||||
|
||||
A persistent immutable template with module lifetime is not generated for now. That approach requires fully validating persistent HashTables, interned strings, nested arrays, MSHUTDOWN, and ZTS, and arrays containing runtime constants or enum cases would still need the request-level path. Until ZendVM's constraints on these combinations are sufficiently validated, the request-level template is the safety boundary. |
||||
|
||||
### 7.2 Changed to fixed property-slot writes |
||||
|
||||
Properties supplemented at runtime already have their class, property name, offset, and type known at compile time. The current implementation reuses the persistent property-offset cache and updates slots via `php::Object::attr(offset)`, already eliminating the property-name hash lookup, the generic write handler, and the Property Hook setter on every object. |
||||
|
||||
It still builds a short-lived `php::Object` carrier for the initializer and reads the offset cache. If profiling later proves this is a hot spot, the final offset can be saved directly after MINIT, or PHPX can add an initialization helper that does not take object ownership. Any further optimization must continue to handle old-value destruction, reference counting, parent-class private slots, Hook backing slots, and exception safety, and must not regress to unprotected raw-pointer assignment. |
||||
|
||||
### 7.3 Enum case can be bound early |
||||
|
||||
An enum case is likewise a refcounted object and cannot directly serve as an internal-class default zval. One could cache the stable enum case pointer or zval in MINIT and then perform correct reference-count copying on each object creation, eliminating the repeated class/case lookup; the object property write itself still cannot be omitted. |
||||
|
||||
### 7.4 Multi-layer allocators on the inheritance chain |
||||
|
||||
When both parent and child classes have runtime defaults, the creation flow delegates layer by layer and runs each initialization, with cost growing with the number of involved inheritance layers. In the future, inheritance chains fully controlled by TypePHP with no special object layout could have their initialization plans merged; built-in-extension parent classes must still call their allocator. |
||||
|
||||
### 7.5 Conservative constants can produce unnecessary allocators |
||||
|
||||
Constants that cannot be resolved in the preprocessing stage conservatively enter the runtime path. A unified constant-default classification pass could be added after symbol preparation to reduce custom allocators for cases that are "actually scalars but unprovable early". That optimization must preserve the distinction between enum cases and array constants. |
||||
|
||||
### 7.6 Dynamic access cost of custom handlers |
||||
|
||||
TypePHP currently installs property handlers for ordinary Zend Classes to support typed-property unset semantics, Property Hooks, and asymmetric write visibility. Installation happens in MINIT and is not equivalent to installing a custom `create_object`; however, dynamic property reads/writes may still enter the handler. Native property accesses already resolved to fixed slots by the compiler must not degrade because of this. |
||||
|
||||
## 8. Regression test requirements |
||||
|
||||
Changes to this flow should at least cover: |
||||
|
||||
- scalars, scalar constant expressions, and empty arrays do not generate a custom allocator; |
||||
- non-empty arrays generate an allocator, and array modifications on two objects do not affect each other; |
||||
- enum case defaults are real enum objects after object creation; |
||||
- classes containing only Property Hooks or asymmetric set visibility do not generate an empty allocator, and Reflection and dynamic read/write behavior do not degrade; |
||||
- Property Hook/asymmetric properties combined with runtime defaults do not trigger setters; |
||||
- when parent and child classes each declare runtime defaults, both parent and child properties are correct; |
||||
- inheriting a built-in extension class does not break its object layout; |
||||
- the exception path restores object handlers; |
||||
- bootstrap compilation and full PHPUnit/PHPT regression pass. |
||||
|
||||
Current core assertions on code generation live in `NewObjectCodegenTest`; runtime semantics are covered by `default-initialization-paths.phpt`, `default-expressions-inheritance.phpt`, and the Property Hook test group. |
||||
@ -0,0 +1,301 @@ |
||||
# TypePHP's Three Object Storage and Passing Models |
||||
|
||||
> Status: current architectural constraint. This document explains why TypePHP simultaneously keeps three object-style value models — Zend Object, PHPX Box, and |
||||
> Native Class Object — along with their respective ownership, passing methods, and boundaries. |
||||
|
||||
## 1. Conclusion |
||||
|
||||
TypePHP currently has three object storage and passing mechanisms: |
||||
|
||||
1. Ordinary PHP/Zend Objects; |
||||
2. PHPX Box, including Std Containers and high-precision types; |
||||
3. `#[Native]` Native Class Objects. |
||||
|
||||
These three are not historical residue of the same design, but separately solve three mutually conflicting problems: |
||||
|
||||
- Zend Object preserves PHP's dynamic object semantics and ZendVM ecosystem compatibility; |
||||
- Box provides an opaque Zend value carrier for C++ types that cannot be fully written into PHP type declarations; |
||||
- Native Class Object provides statically-knowable business objects with a fixed layout close to C/C++, raw-pointer calls, and |
||||
tracing GC. |
||||
|
||||
None of these mechanisms can replace the other two without losing a core capability of the others. The current design explicitly accepts |
||||
the long-term coexistence of the three models and does not target a "unified object representation". |
||||
|
||||
## 2. Overview |
||||
|
||||
| Dimension | Zend Object | PHPX Box | Native Class Object | |
||||
| --- | --- | --- | --- | |
||||
| Typical value | Ordinary PHP class instance | Std Container, BigInt, BigFloat, Decimal | `#[Native] class` instance | |
||||
| Primary representation | `zend_object` / zval | `zend_resource` + `php::Box *` | C++ struct in Native Heap + raw pointer | |
||||
| Type identity | `zend_class_entry *` | Box C++ dynamic type, `type_info`/type ID | Compile-time Native class, dynamic type saved in descriptor | |
||||
| Lifecycle | Zend reference counting + Zend cycle GC | Zend resource reference counting calling the Box destructor | Wren-style precise, non-moving mark-sweep GC | |
||||
| Argument passing | `php::Object` / `php::Var`, copying the handle and adjusting RC | `php::Var` carrying the resource; hot paths extract the concrete C++ reference | Concrete `NativeClass *` passed by value, without adjusting RC | |
||||
| Property/method access | Zend handlers, dynamic lookup, or already-cached Native Call | The compiler generates operations based on the concrete Box type | Fixed-offset field access and definite `php_*` Native Call | |
||||
| Dynamic PHP interop | Complete | Limited interop as an opaque resource | Cannot enter the ZendVM value boundary | |
||||
| Cyclic graph handling | Zend GC can scan the Zend object graph | Zend GC does not scan the C++ object graph inside Box | The Native descriptor precisely traces the Native pointer graph | |
||||
| Core goal | PHP compatibility | Carrying C++ generic/extended values | Extreme static performance | |
||||
|
||||
## 3. Ordinary PHP/Zend Object |
||||
|
||||
### 3.1 Storage |
||||
|
||||
Ordinary classes are registered with the ZendVM, and instances are represented by `zend_object`. TypePHP holds the corresponding zval through PHPX RAII types such as `php::Object`, |
||||
`php::Variant`/`php::Var`. |
||||
|
||||
The object has Zend's class entry, property table, object handlers, and method metadata. Based on compile-time information, |
||||
TypePHP can optimize some accesses into definite Native Calls, but the object identity and lifecycle still belong to the ZendVM. |
||||
|
||||
### 3.2 Passing and Lifecycle |
||||
|
||||
PHP object assignment and argument passing copy the object handle, not the object entity, and follow Zend reference counting. Cyclic references in the object graph |
||||
are handled by Zend GC. Objects can naturally enter: |
||||
|
||||
- PHP arrays and ordinary object properties; |
||||
- `mixed`/`object` variables; |
||||
- Closures, Generators, Fibers, and dynamic calls; |
||||
- Reflection, serialization, and extension functions; |
||||
- PHP code executed by the ZendVM. |
||||
|
||||
### 3.3 Why It Must Be Kept |
||||
|
||||
Only Zend Object can fully carry PHP's runtime object semantics. Replacing it with Box would lose the class entry, object |
||||
handlers, visibility, Reflection, and dynamic dispatch; replacing it with Native Object would lose ZendVM visibility, |
||||
and force all dynamic behavior to degrade to compile-time restrictions. |
||||
|
||||
Ordinary PHP classes therefore always use Zend Object. The compiler can optimize calls, but cannot change its object model. |
||||
|
||||
## 4. PHPX Box |
||||
|
||||
### 4.1 Storage |
||||
|
||||
`php::Box` is a C++ polymorphic base class managed by PHPX. The Box pointer is registered as a Zend resource and carried by |
||||
`php::Var`: |
||||
|
||||
```text |
||||
zval(IS_RESOURCE) |
||||
-> zend_resource |
||||
-> php::Box* |
||||
-> concrete C++ value |
||||
``` |
||||
|
||||
The Zend resource's destructor callback ultimately calls `Box::destroy()`. Box can therefore pass through ordinary zval/Variant |
||||
call boundaries while hiding the concrete C++ type that Zend cannot express. |
||||
|
||||
Current main users include: |
||||
|
||||
- `StdContainerBox<std::vector<T>>`; |
||||
- `StdContainerBox<std::array<T, N>>`; |
||||
- `StdContainerBox<map-like type>`; |
||||
- High-precision values such as BigInt, BigFloat, and Decimal. |
||||
|
||||
### 4.2 The Std Container Hot Path |
||||
|
||||
Std Container local variables have a two-layer representation: |
||||
|
||||
```cpp |
||||
php::Var values = php::Var(new php::StdContainerBox<Container>(type_id)); |
||||
auto &values_ref = values.toBox<php::StdContainerBox<Container>>()->container; |
||||
``` |
||||
|
||||
`php::Var` is responsible for the lifecycle and necessary boundary passing; the concrete container reference is used for subsequent element access, avoiding re-extracting the Box on every operation. The container's key/value/length and other generic information are jointly saved by the compiler and the concrete C++ template type. |
||||
|
||||
When a Std Container is passed across TypePHP functions, the PHP function signature cannot express the following C++ type information: |
||||
|
||||
```text |
||||
std::vector<int> |
||||
std::vector<string> |
||||
std::map<string, App\User> |
||||
``` |
||||
|
||||
A PHP parameter can at most declare a non-generic class name or pseudo-type; it cannot simultaneously carry the container kind, key type, value |
||||
type, array dimensions, and length. The current approach uses `UnsafePtr`/`std::unsafe_cast()` with compiler type ID checking, |
||||
rather than generating every combination as a PHP class. |
||||
|
||||
In theory, parameter and return value annotations could be added to describe generics, but this would require maintaining extra metadata at every declaration, call, return, property, and propagation point, |
||||
and PHP Reflection still cannot fully express it. This standalone generic ABI is not being introduced for now. |
||||
|
||||
### 4.3 Box Boundaries |
||||
|
||||
Box is an opaque value carrier, not a general-purpose object system: |
||||
|
||||
- Zend GC only sees the resource and does not scan C++ references held inside Box; |
||||
- Box does not provide PHP class method tables, property tables, inheritance, or Reflection; |
||||
- The concrete type is recovered through `dynamic_cast`, type ID, or dedicated helpers; |
||||
- Box should not be used to build arbitrary cyclic object graphs that require bidirectional Zend/Box tracing; |
||||
- The usable locations and escape paths of Std Container continue to be restricted by the compiler. |
||||
|
||||
Box is suitable for numeric values, containers, and other extension values with clear boundaries. It is not suitable for replacing Native |
||||
business objects with arbitrary field reference relationships. |
||||
|
||||
### 4.4 Why It Must Be Kept |
||||
|
||||
The generic types of Std Container cannot be fully expressed by PHP function parameters; high-precision values in turn need to participate in existing operations and calls as |
||||
`php::Var`. Box provides all of the following: |
||||
|
||||
- A stable carrier that can be placed into a zval; |
||||
- Runtime recovery of the concrete C++ type; |
||||
- Automatic destruction within the Zend request lifecycle; |
||||
- A lightweight implementation that does not register a PHP class for each template instantiation. |
||||
|
||||
Zend Object cannot directly express C++ template instances; Native raw pointers cannot safely cross `php::Var` and dynamic |
||||
ZendVM boundaries. Therefore Box still has a reason to exist independently. |
||||
|
||||
## 5. Native Class Object |
||||
|
||||
### 5.1 Storage |
||||
|
||||
`#[Native]` classes do not register a Zend class, do not generate Zend object handlers, and have no zval representation. Each |
||||
object is a fixed-layout C++ struct in the Native Heap; TypePHP local variables, parameters, return values, and fields hold |
||||
concrete Native pointers: |
||||
|
||||
```cpp |
||||
php_app__point *point; |
||||
``` |
||||
|
||||
Methods continue to use TypePHP's free-function ABI: |
||||
|
||||
```cpp |
||||
php::Float php_app__point__length(php_app__point &this_); |
||||
``` |
||||
|
||||
Ordinary calls only pass a pointer value. No zval is created, no resource is registered, no reference counting is performed, and nothing goes through |
||||
`zend_call_function()`. |
||||
|
||||
### 5.2 Lifecycle |
||||
|
||||
Native Objects use an independent Wren-style precise, non-moving, stop-the-world mark-sweep GC in PHPX: |
||||
|
||||
- Native local variables, parameters, return temporaries, and global/static slots enter a precise root frame; |
||||
- The Native object descriptor is responsible for tracing Native pointer fields; |
||||
- When a Std Container saves a Native pointer, a dedicated container root frame is registered; |
||||
- Cyclic references are collected by the tracing GC, without relying on reference counts dropping to zero; |
||||
- The 16-byte GC header saves the minimal state required by the collector; |
||||
- `__destruct()` is executed by Native finalization, not by the Zend object destructor. |
||||
|
||||
Native pointer assignment does not increase the reference count and does not need a write barrier. Fixed fields are accessed directly by C++ offset. |
||||
|
||||
### 5.3 Passing Boundaries |
||||
|
||||
Native Object parameters and return values must explicitly declare a concrete Native class, or a supported nullable concrete type: |
||||
|
||||
```php |
||||
function distance(Point $left, Point $right): float; |
||||
function findPoint(): ?Point; |
||||
``` |
||||
|
||||
This lets the compiler generate the signature directly as `Point *`. Native Objects do not support: |
||||
|
||||
- Passing to PHP/ZendVM functions, Closures, or dynamic callables; |
||||
- Saving into PHP arrays, ordinary Zend Object properties, or `mixed`; |
||||
- Automatic conversion to `php::Object`, `php::Var`, or Interface value; |
||||
- Recovering the type through the runtime class name; |
||||
- Using the generic PHPX `toObject()` helper to complete boxing or unboxing. Native Classes can declare their own |
||||
`toObject(): object` method; keyword calls resolve directly to that Native Call and do not provide a generic bridge. |
||||
|
||||
When entering the PHP API, the user must explicitly convert the data, for example first calling Native `toArray(): array`, then passing |
||||
the result to `json_encode()`. This conversion produces a data copy and does not preserve the Native object identity. |
||||
|
||||
### 5.4 Why It Must Be Kept |
||||
|
||||
The goal of Native Class is hot-path performance close to C/C++: |
||||
|
||||
- A one-machine-word object handle; |
||||
- Fixed field layout; |
||||
- No Zend RC increment/decrement; |
||||
- No `zend_object` or `zend_resource` carrier allocation; |
||||
- Definite-symbol Native Calls; |
||||
- Inlinable and devirtualizable by the C++ compiler. |
||||
|
||||
If Box were used instead, each Native Object would need resource/zval wrapping, RC management, and concrete type recovery, and |
||||
Zend GC cannot scan the Native pointer graph inside Box; this both reduces performance and cannot correctly replace Native tracing |
||||
GC. If a custom `zend_object` were used instead, although it could connect to Zend GC and dynamic boundaries, the object header, RC, |
||||
handlers, and access paths would all change the performance positioning of Native Class. |
||||
|
||||
Therefore Native Class continues to use an independent Native Heap and a raw-pointer ABI. |
||||
|
||||
## 6. Why They Cannot Be Unified |
||||
|
||||
### 6.1 They Cannot All Become Zend Object |
||||
|
||||
This would unify dynamic semantics, but it would make Std Container generic instances and Native Class both bear the Zend object |
||||
header, RC, handlers, class registration, and dynamic access costs. Native Class would no longer be close to C/C++, |
||||
and Std Container would need a runtime class system designed for a large number of template combinations. |
||||
|
||||
### 6.2 They Cannot All Become Box |
||||
|
||||
Box can carry C++ values through zval, but Zend GC does not understand the object graph inside Box. It cannot replace the dynamic metadata of ordinary PHP |
||||
Object, nor can it provide a raw-pointer hot path while retaining Native cycle collection capability. |
||||
|
||||
### 6.3 They Cannot All Become Native Pointers |
||||
|
||||
Native pointers require complete static typing. Ordinary PHP objects need Reflection, dynamic properties, dynamic callables, |
||||
and Zend extension interop; the complete generic types of Std Container cannot be written into PHP parameter signatures. Turning these values into |
||||
raw pointers would produce type erasure that cannot be proven safe statically, and could lead to incorrect pointer conversion and crashes. |
||||
|
||||
### 6.4 No Automatic Bridging |
||||
|
||||
There is no implicit object identity conversion among the three models. Automatic boxing/unboxing would hide allocation, copying, RC, and GC root |
||||
changes, and would also make compiler boundaries no longer reliable. |
||||
|
||||
Allowed conversions must have clear semantics: |
||||
|
||||
- Std Container to PHP array: copies container data; |
||||
- Entity methods of Native Object such as `toArray()`: defined by the user and explicitly copy data; |
||||
- Explicit scalar conversion of high-precision types: produces new PHP scalar values; |
||||
- Ordinary Zend Object does not automatically become a Native Object. |
||||
|
||||
## 7. Compiler Implementation Constraints |
||||
|
||||
Future changes must preserve the following invariants: |
||||
|
||||
1. Determine the object model from the static type first, then choose the code generation path; never guess one of the three at runtime. |
||||
2. Native Objects must not be wrapped into `php::Var` or passed into the ZendVM due to a generic fallback. |
||||
3. Box concrete type recovery must validate the resource type and the concrete C++ type / type ID. |
||||
4. Zend Object optimization must not change Zend object identity, lifecycle, or dynamic visibility. |
||||
5. The argument ABIs of the three models must not be mixed: `php::Object`, Box-bearing `php::Var`, and `NativeClass *` |
||||
respectively represent different ownership and type constraints. |
||||
6. Cross-model conversions must be explicit, and the allocation or copying cost must be reflected in documentation and generated code. |
||||
7. If a new feature requires sacrificing all Native Class hot paths to gain a small amount of dynamic compatibility, it should be prohibited at compile time first. |
||||
8. If a new C++ generic type needs to cross the Zend value boundary, Box should be evaluated first, rather than widening the dynamic |
||||
boundary of Native Object. |
||||
9. If a value needs complete PHP object semantics, Zend Object should be used, and Box must not be treated as a simplified PHP class. |
||||
|
||||
## 8. Code Locations |
||||
|
||||
Main implementation entry points: |
||||
|
||||
```text |
||||
Ordinary Zend Object |
||||
compiler/src/Parser/* |
||||
phpx/include/phpx.h Object / Variant / Zend API wrappers |
||||
|
||||
PHPX Box and Std Container |
||||
phpx/include/phpx.h Box / StdContainerBox<T> |
||||
phpx/src/core/base.cc Box resource registration and destructor |
||||
compiler/src/Parser/StdContainerTrait.php |
||||
|
||||
Native Class Object |
||||
compiler/src/NativeClass/ |
||||
compiler/src/Transform/NativeClassAttributeLowering.php |
||||
phpx/include/phpx_native_gc.h |
||||
phpx/src/core/native_gc.cc |
||||
phpx/thirdparty/wren-gc/ |
||||
``` |
||||
|
||||
Detailed rules are in [STD_CONTAINERS.md](STD_CONTAINERS.md), |
||||
[NATIVE_CLASS_OBJECT.md](NATIVE_CLASS_OBJECT.md), and |
||||
[NATIVE_CLASS_IMPLEMENTATION_AUDIT.md](NATIVE_CLASS_IMPLEMENTATION_AUDIT.md). |
||||
|
||||
## 9. Current Decision |
||||
|
||||
The following refactorings are not being implemented at the current stage: |
||||
|
||||
- Not removing Wren GC; |
||||
- Not changing Native Object to Box or a custom Zend Object; |
||||
- Not adding a generic `toObject()` dynamic recovery mechanism to Native Object; the Native Class custom |
||||
`toObject(): object` remains an ordinary definite Native Call; |
||||
- Not changing Std Container to a raw-pointer ABI whose type cannot be expressed across signatures; |
||||
- Not attempting to cover the three object models with a single unified wrapper. |
||||
|
||||
These boundaries will be re-evaluated in the future only when the PHP language layer can stably express generic parameters, or when a new ABI |
||||
that has passed benchmark and complete GC correctness validation emerges. Until then, the coexistence of the three mechanisms is an intentional architectural choice. |
||||
@ -0,0 +1,406 @@ |
||||
# Patent Application Technical Disclosure: A Method for Implementing Strongly-Typed Data Containers in a Dynamic Language Using C++ Templates |
||||
|
||||
> This document is a draft technical disclosure for a patent application, intended to explain the technical solution to a patent agent. In this document, "the present invention" refers to "a method for implementing strongly-typed data containers in a dynamic language using C++ templates." This document does not constitute legal advice; the formal claims should be further drafted by a patent agent based on search results. |
||||
|
||||
## 1. Technical Application Product |
||||
|
||||
The present invention is applied to the Swoole-Compiler PHP AOT compiler. This product is used to pre-compile PHP dynamic-language programs into C++ native code, PHP extensions, or executable programs, while preserving PHP runtime compatibility in the compiled program. |
||||
|
||||
The present invention focuses on solving the problems of indeterminate array and container structure types, high runtime overhead, and difficulty in leveraging C++ static-type optimizations in dynamic languages. The solution introduces strongly-typed container declarations at the dynamic-language syntax level and converts them into C++ template container instances during the AOT compilation stage, enabling dynamic-language programs to achieve data-structure performance close to that of a static language in localized performance hotspots, while maintaining interoperability with dynamic-language arrays. |
||||
|
||||
## 2. Terminology |
||||
|
||||
| Term | English Explanation | |
||||
| --- | --- | |
||||
| PHP | A dynamically-typed scripting language | |
||||
| AOT | Ahead-Of-Time; compiling to target code before the program runs | |
||||
| C++ Template | A C++ mechanism for generating strongly-typed code at compile time | |
||||
| Dynamic language | A language in which variable types and function call targets can change at runtime | |
||||
| Strongly-typed container | A container whose key, value, length, nested structure, or class constraints are determined at compile time | |
||||
| PHP Array | The built-in array of the PHP language, which combines list, dictionary, and hash table semantics | |
||||
| zval | The internal structure used by the PHP runtime to store a value of any type | |
||||
| HashTable | The common underlying hash table structure of PHP Array | |
||||
| AST | Abstract Syntax Tree | |
||||
| Meta-information | Information recorded by the compiler about container kind, type, dimension, C++ declaration, etc. | |
||||
| Type identifier | An integer identifier used to distinguish different strongly-typed container structures | |
||||
| UnsafePtr | A controlled pointer wrapper that carries a container pointer and a type identifier | |
||||
|
||||
## 3. Technical Background and Existing Technical Solutions |
||||
|
||||
### 3.1 Broad Technical Background |
||||
|
||||
The advantage of dynamic languages lies in development flexibility. Taking PHP as an example, the same variable can hold an integer, a floating-point number, a string, an array, or an object in different places. The same PHP Array can act as a contiguous list, as a dictionary with string keys, and can simultaneously mix integer keys, string keys, and values of different types. |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
$data = []; |
||||
$data[] = 1; |
||||
$data["name"] = "swoole"; |
||||
$data[10] = new stdClass(); |
||||
``` |
||||
|
||||
This design lowers the barrier to business development, but it also makes it difficult for the compiler to determine the real shape of a data structure at compile time. For an AOT compiler, if the array element type, key type, length, and nested structure cannot be determined, it can only conservatively generate generic dynamic container code and cannot fully leverage C++ static typing and template optimization capabilities. |
||||
|
||||
In contrast, static languages such as C++, Rust, and Go generally require containers to have explicit types, for example `std::vector<int>` and `std::array<double, 100>`. Such containers have element sizes, access patterns, and memory layouts determined at compile time, so the access path is short and the compiler can further perform inlining, register allocation, and loop optimization. |
||||
|
||||
The present invention attempts to establish a localized strongly-typed data container mechanism between dynamic and static languages, so that dynamic-language code still keeps array-like syntax but is converted into C++ template containers at compile time. |
||||
|
||||
### 3.2 Narrow Technical Background |
||||
|
||||
Swoole-Compiler is a PHP AOT compiler. It parses PHP source files into an abstract syntax tree, then generates C++ code and compiles it into a PHP extension or a binary program. For ordinary PHP variables, the compiler can use runtime wrappers such as `php::Var` and `php::Array` to represent dynamic semantics. |
||||
|
||||
However, for scenarios such as high-frequency array access, numerical computation, fixed-length buffers, mapping tables, and object collections, continuing to use ordinary PHP Array incurs the following overhead: |
||||
|
||||
- Both keys and values require dynamic type judgment; |
||||
- Each value usually needs to be represented by a zval; |
||||
- Hash lookup, reference counting, copy-on-write, and other mechanisms lengthen the runtime path; |
||||
- The memory layout is not contiguous, resulting in poor CPU cache utilization; |
||||
- It is difficult for the compiler to confirm whether an array stores only one specific type; |
||||
- Passing large containers across functions tends to cause copying or dynamic wrapping overhead. |
||||
|
||||
Therefore, a strongly-typed container expression, checking, and code generation scheme suitable for an AOT compiler is needed. |
||||
|
||||
### 3.3 Closest Existing Technical Solutions |
||||
|
||||
Existing technologies can be roughly classified into the following categories: |
||||
|
||||
1. **Dynamic container solution using PHP Array exclusively** |
||||
All arrays are represented by the PHP runtime HashTable and zval. This solution offers good compatibility, but performance is limited by dynamic typing and the hash structure. |
||||
|
||||
2. **User-handwritten C++ extension solution** |
||||
Developers handwrite structures such as `std::vector` and `std::map` in C++ and expose them to PHP through PHP extension functions. This solution offers high performance, but the development cost is high, and users must manually maintain the type mapping between PHP and C++. |
||||
|
||||
3. **Generic JIT or AOT type inference solution** |
||||
The compiler attempts to infer the element types of a PHP Array from context. However, the dynamic writes, dynamic keys, function parameters, and return values of PHP Array make inference unstable, so optimization is usually only possible in very localized scenarios. |
||||
|
||||
4. **PHP userland container class solution** |
||||
For example, wrapping arrays or specific data structures through object classes. This solution improves the interface specification, but the underlying implementation may still rely on PHP objects, zval, and dynamic method calls, making it difficult to reach the performance level of C++ template containers. |
||||
|
||||
## 4. Shortcomings of Existing Technologies and Objectives of the Present Invention |
||||
|
||||
### 4.1 Shortcomings of Existing Technologies |
||||
|
||||
Existing solutions have the following shortcomings: |
||||
|
||||
1. The indeterminate types of dynamic arrays prevent the compiler from stably generating strongly-typed target code. |
||||
2. The underlying structure of ordinary PHP Array is too generic, incurring hash lookup and dynamic typing overhead in high-frequency access. |
||||
3. Handwritten C++ extensions require users to understand PHP extension development, memory management, and type conversion, raising the development barrier. |
||||
4. Generic type inference finds it hard to express complete container information such as fixed length, nested dimension, key type, and value class constraints. |
||||
5. There is a lack of unified interoperability rules between strongly-typed containers and PHP Array, easily splitting performance from compatibility. |
||||
6. When container references are passed across functions, directly exposing C++ pointers creates risks of type misuse and memory unsafety. |
||||
|
||||
### 4.2 Objectives of the Present Invention |
||||
|
||||
The objective of the present invention is to provide a method for implementing strongly-typed data containers in a dynamic language using C++ templates, so that dynamic-language programs can declare strongly-typed containers in localized code, the compiler generates C++ template container code, and automatic conversion to and from dynamic-language arrays is performed when necessary. |
||||
|
||||
Further, the present invention also provides a container reference passing mechanism carrying a type identifier, so that strongly-typed containers can be passed between Native functions with low copy cost while runtime type consistency checking is performed. |
||||
|
||||
## 5. Technical Solution of the Present Invention |
||||
|
||||
### 5.1 Overall Solution |
||||
|
||||
The present invention defines a set of strongly-typed container construction syntax in the dynamic language, for example: |
||||
|
||||
```php |
||||
$a = std::array(Type::Int, 100); |
||||
$v = std::vector(Type::Float); |
||||
$m = std::ordered_map(Type::String, Type::Int); |
||||
$h = std::map(Type::Int, User::class); |
||||
``` |
||||
|
||||
The compiler recognizes these construction expressions during the AOT stage, generates container meta-information, and converts them into C++ template instances: |
||||
|
||||
```cpp |
||||
php::StdArray<php::Int, 100> a{}; |
||||
php::StdVector<php::Float> v{}; |
||||
php::StdOrderedMap<php::Str, php::Int> m{}; |
||||
php::StdMap<php::Int, php::Object> h{}; |
||||
``` |
||||
|
||||
The compiler subsequently uses this meta-information for static checking and code generation during subscript access, assignment, iteration, function argument passing, and type conversion. |
||||
|
||||
### 5.2 System Composition |
||||
|
||||
```text |
||||
Figure 1: System composition diagram |
||||
|
||||
PHP source input module |
||||
| |
||||
v |
||||
Abstract syntax tree parsing module |
||||
| |
||||
v |
||||
Strongly-typed container recognition module |
||||
| |
||||
v |
||||
Container meta-information construction module |
||||
| |
||||
+--> Type identifier registration module |
||||
| |
||||
+--> Subscript access code generation module |
||||
| |
||||
+--> Assignment and copy determination module |
||||
| |
||||
+--> PHP Array interoperability module |
||||
| |
||||
+--> UnsafePtr auto-boxing and checking module |
||||
| |
||||
v |
||||
C++ template code generation module |
||||
| |
||||
v |
||||
C++ compiler |
||||
| |
||||
v |
||||
PHP extension or binary program |
||||
``` |
||||
|
||||
Each module is described as follows: |
||||
|
||||
- PHP source input module: reads dynamic-language source files. |
||||
- Abstract syntax tree parsing module: parses source code into syntax tree nodes. |
||||
- Strongly-typed container recognition module: recognizes container construction expressions such as `std::array`. |
||||
- Container meta-information construction module: records the container kind, key type, value type, class constraints, dimensions, etc. |
||||
- Type identifier registration module: generates a comparable type identifier for each container structure. |
||||
- Subscript access code generation module: generates array access, bounds checking, and key conversion code based on the container type. |
||||
- Assignment and copy determination module: determines whether to perform a C++ container copy or convert to PHP Array. |
||||
- PHP Array interoperability module: generates `php::toArray()` at dynamic semantic boundaries. |
||||
- UnsafePtr auto-boxing and checking module: generates container pointer wrappers carrying type identifiers at function call boundaries. |
||||
- C++ template code generation module: outputs strongly-typed C++ template container code. |
||||
|
||||
### 5.3 Method Flow |
||||
|
||||
```text |
||||
Figure 2: Method flow diagram |
||||
|
||||
Step S1: Parse the dynamic-language source code to obtain an abstract syntax tree; |
||||
Step S2: Recognize strongly-typed container construction expressions; |
||||
Step S3: Parse the container kind, key type, value type, class constraints, and dimensions; |
||||
Step S4: Generate container meta-information and register the type identifier; |
||||
Step S5: Record in the variable table that the variable is a strongly-typed container; |
||||
Step S6: When a subscript access is encountered, generate strongly-typed access code based on the container meta-information; |
||||
Step S7: When an assignment is encountered, determine whether the left and right sides are the same strongly-typed container; |
||||
Step S8: If they are the same, generate a C++ container copy; if not, report an error or convert to PHP Array according to the rules; |
||||
Step S9: When a Native function UnsafePtr parameter is encountered, auto-box the container pointer and type identifier; |
||||
Step S10: When the callee unboxes, check the type identifier; if it matches, return a C++ reference; otherwise, throw an exception. |
||||
``` |
||||
|
||||
### 5.4 Container Meta-information |
||||
|
||||
Container meta-information includes at least the following fields: |
||||
|
||||
```text |
||||
kind: container kind, for example array, vector, map, ordered_map; |
||||
decl: target C++ template declaration; |
||||
type: C++ type of the value; |
||||
class: class name when the value is an object; |
||||
keyType: key type of map or ordered_map; |
||||
sizes: dimension array of std::array; |
||||
bytes: estimated memory size of std::array; |
||||
typeId: type identifier generated from the above fields. |
||||
``` |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
$b = std::array(std::array(Type::Int, 3), 2); |
||||
``` |
||||
|
||||
The corresponding meta-information can be represented as: |
||||
|
||||
```text |
||||
kind=array |
||||
decl=php::StdArray<php::StdArray<php::Int, 3>, 2> |
||||
type=php::Int |
||||
sizes=[3, 2] |
||||
bytes=2 * 3 * sizeof(php::Int) |
||||
typeId=automatically assigned integer |
||||
``` |
||||
|
||||
### 5.5 std::array Nested Type Derivation |
||||
|
||||
`std::array` supports nested structures. The present invention derives the sub-array type based on the access level. |
||||
|
||||
Example: |
||||
|
||||
```php |
||||
$a = std::array(Type::Int, 3); |
||||
$b = std::array(std::array(Type::Int, 3), 2); |
||||
$a = $b[1]; |
||||
``` |
||||
|
||||
Processing method: |
||||
|
||||
1. The compiler reads the dimension information `[2, 3]` of `$b`. |
||||
2. Parse the access level of `$b[1]` as 1. |
||||
3. Compute the remaining dimensions `[3]`. |
||||
4. Derive the type of `$b[1]` as `std::array<int, 3>`. |
||||
5. Compare it with the type of `$a`. |
||||
6. If they are exactly the same, generate a C++ copy: |
||||
|
||||
```cpp |
||||
a = b[php::safeIndex(php::toInt(1L), 2)]; |
||||
``` |
||||
|
||||
Here `safeIndex` denotes the bounds-checking function. It ensures that dynamic-language subscript access still preserves out-of-bounds checking semantics. |
||||
|
||||
### 5.6 Same-type Copy and Dynamic Array Conversion |
||||
|
||||
The present invention divides assignment into two categories: |
||||
|
||||
The first category: the left value is a strongly-typed container and the right value is an exactly identical strongly-typed container: |
||||
|
||||
```php |
||||
$a = std::vector(Type::Int); |
||||
$b = std::vector(Type::Int); |
||||
$a = $b; |
||||
``` |
||||
|
||||
Generated: |
||||
|
||||
```cpp |
||||
a = b; |
||||
``` |
||||
|
||||
The second category: the left value is an ordinary dynamic variable and the right value is a strongly-typed container: |
||||
|
||||
```php |
||||
$arr = $a; |
||||
``` |
||||
|
||||
Generated: |
||||
|
||||
```cpp |
||||
arr = php::toArray(a); |
||||
``` |
||||
|
||||
This rule keeps C++ performance within strongly-typed regions, and automatically turns strongly-typed containers into ordinary PHP Array when they flow into dynamic-language regions. |
||||
|
||||
### 5.7 Subscript Access and Writing |
||||
|
||||
For `std::vector`: |
||||
|
||||
```php |
||||
$v[] = 1; |
||||
$v[0] = 2; |
||||
``` |
||||
|
||||
Generated similarly to: |
||||
|
||||
```cpp |
||||
v.push_back(php::toInt(1L)); |
||||
v.offsetSet(php::toInt(0L), php::toInt(2L)); |
||||
``` |
||||
|
||||
For `std::map`: |
||||
|
||||
```php |
||||
$m["x"] = 10; |
||||
``` |
||||
|
||||
Generated similarly to: |
||||
|
||||
```cpp |
||||
m.offsetSet(php::toString("x"), php::toInt(10L)); |
||||
``` |
||||
|
||||
For class-typed values: |
||||
|
||||
```php |
||||
$v = std::vector(User::class); |
||||
$v[] = new User(); |
||||
``` |
||||
|
||||
The compiler checks whether the written object is of the specified class, avoiding mixing incorrect objects into the container. |
||||
|
||||
### 5.8 foreach Iteration |
||||
|
||||
Strongly-typed container iteration is compiled into a C++ iterator loop: |
||||
|
||||
```php |
||||
foreach ($v as $i => $value) { |
||||
// ... |
||||
} |
||||
``` |
||||
|
||||
Generated similarly to: |
||||
|
||||
```cpp |
||||
for (auto it = v.begin(); it != v.end(); ++it) { |
||||
i = it - v.begin(); |
||||
value = *it; |
||||
} |
||||
``` |
||||
|
||||
For map types, the key is obtained from `it->first` and the value from `it->second`. |
||||
|
||||
### 5.9 UnsafePtr Auto-boxing and Type Checking |
||||
|
||||
To support low-copy passing of containers between Native functions, the present invention provides the `UnsafePtr` parameter mechanism. |
||||
|
||||
User code: |
||||
|
||||
```php |
||||
function update(UnsafePtr $ptr): void |
||||
{ |
||||
$v = std::unsafe_cast(std::vector(Type::Int), $ptr); |
||||
$v[0] = 100; |
||||
} |
||||
|
||||
function main(): void |
||||
{ |
||||
$v = std::vector(Type::Int, 1); |
||||
update($v); |
||||
} |
||||
``` |
||||
|
||||
Caller-side generation: |
||||
|
||||
```cpp |
||||
php_update(php_create_unsafe_ptr(&v, typeId)); |
||||
``` |
||||
|
||||
Callee-side generation: |
||||
|
||||
```cpp |
||||
auto &v = php_unsafe_cast<php::StdVector<php::Int>>(ptr, typeId); |
||||
``` |
||||
|
||||
Where `UnsafePtr` holds: |
||||
|
||||
```cpp |
||||
void *ptr; |
||||
uint32_t type_id; |
||||
``` |
||||
|
||||
When unboxing, `type_id` is compared; a C++ reference is returned only if it matches, otherwise a type exception is thrown. This avoids incorrectly converting `std::vector<int>` to `std::vector<float>`. |
||||
|
||||
## 6. Key Points and Points to Protect |
||||
|
||||
1. Recognize strongly-typed container construction syntax during the AOT compilation of a dynamic language, and convert it into C++ template container instances. |
||||
2. Use a container meta-information table to uniformly record the container kind, C++ declaration, key type, value type, class constraints, dimensions, memory size, and type identifier. |
||||
3. Derive the sub-array type of nested `std::array` based on the access level, and support C++ copy between sub-arrays and same-type containers. |
||||
4. Determine assignment semantics based on the left and right container meta-information: if exactly the same, generate a C++ copy; otherwise, convert to a dynamic array or report a compile-time error. |
||||
5. Generate strongly-typed C++ access code for subscript access while preserving the dynamic language's bounds-checking or key type conversion semantics. |
||||
6. Generate a C++ iterator loop for foreach while preserving the dynamic language's key/value iteration form. |
||||
7. At Native function call boundaries, auto-box strongly-typed containers into UnsafePtr carrying type identifiers based on ArgInfo. |
||||
8. Perform runtime checking based on the type identifier during unboxing, and return a C++ container reference only after the check passes. |
||||
|
||||
## 7. Advantages Compared with Existing Technologies |
||||
|
||||
Compared with the ordinary PHP Array solution, the present invention can determine the container structure and element types at compile time and generate C++ template instances, thereby reducing dynamic type judgment, hash lookup, and zval wrapping overhead. |
||||
|
||||
Compared with the handwritten C++ extension solution, developers still use syntax close to PHP arrays; container declaration, type checking, subscript access, copy, iteration, dynamic array conversion, and cross-function reference passing are all completed automatically by the compiler, lowering the development barrier. |
||||
|
||||
Compared with ordinary type inference solutions, the present invention does not rely on guessing how a PHP Array is used; instead, it establishes stable type meta-information through explicit strongly-typed container syntax, making the compilation result more predictable. |
||||
|
||||
## 8. Optional Implementations |
||||
|
||||
The present invention is not limited to the PHP language; it can also be used in AOT compilers for Python, JavaScript, Ruby, and other dynamic languages. As long as a dynamic-language compiler can recognize strongly-typed container declarations and generate C++, Rust, Go, or other static-language target code, a similar technical solution can be adopted. |
||||
|
||||
The C++ template containers in the present invention are also not limited to `StdArray`, `StdVector`, `StdOrderedMap`, and `StdMap`; they can be extended to strongly-typed containers such as queues, sets, ring buffers, matrices, and tensors. |
||||
|
||||
## 9. Confidentiality Statement |
||||
|
||||
This document involves the internal compiler implementation of Swoole-Compiler, the design of strongly-typed container meta-information, the UnsafePtr type identifier mechanism, and code generation strategies. Before formal filing, it is recommended that it be managed as internal technical material. |
||||
@ -0,0 +1,453 @@ |
||||
# Patent Application Technical Disclosure: A Method and System for Compilation and Execution of a Hybrid-State Programming Language |
||||
|
||||
> This document is a draft technical disclosure for a patent application, explaining the technical solution to a patent agent. "The present invention" herein refers to "a method and system for compilation and execution of a hybrid-state programming language". This document does not constitute legal advice; the formal claims should be further drafted by a patent agent in combination with search results. |
||||
|
||||
## 1. Technical Application Product |
||||
|
||||
The present invention is applied to the Swoole-Compiler PHP AOT compiler. Swoole-Compiler neither simply fully staticizes PHP, nor is it a pure interpreter or pure JIT compiler in the traditional sense, but rather a "hybrid-state programming language compilation and execution system". |
||||
|
||||
In this system, the same PHP program can simultaneously contain: |
||||
|
||||
- Static-state code that can be statically analyzed and compiled ahead of time; |
||||
- Dynamic-state code that needs to preserve PHP's dynamic semantics; |
||||
- Boundary transition code between static state and dynamic state; |
||||
- Entry points that can be called by the dynamic runtime as a PHP extension; |
||||
- Entry points that can be executed as a standalone binary program. |
||||
|
||||
This system enables a dynamic language to gain the performance advantages of a static language while preserving the flexibility advantages of a dynamic language. |
||||
|
||||
## 2. Terminology |
||||
|
||||
| Term | Description | |
||||
| --- | --- | |
||||
| Hybrid state | A single program simultaneously contains static compilation state and dynamic interpretation state, and can switch between the two states | |
||||
| Static state | A state where the compiler can determine types, functions, methods, properties, or control paths at compile time | |
||||
| Dynamic state | A state where types, functions, methods, or properties must be resolved at runtime according to PHP semantics | |
||||
| AOT | Ahead-Of-Time compilation | |
||||
| VM | Virtual Machine; here it refers to the PHP Zend VM | |
||||
| Native function | A PHP function or method compiled by AOT into a C++ function | |
||||
| Dynamic call | Looking up and calling a target at runtime based on function name, method name, object type, or callback object | |
||||
| Static direct link | Determining the call target at compile time and generating a direct C++ function call | |
||||
| ArgInfo | Parameter information, including type, default value, reference, variadic parameters, etc. | |
||||
| Symbol table | A data structure in which the compiler records information about functions, classes, interfaces, traits, constants, properties, methods, etc. | |
||||
| Wrapper function | A bridging function that converts a Zend VM call entry point into a Native function call | |
||||
| Fallback | Falling back to the dynamic runtime path when static analysis cannot guarantee semantic correctness | |
||||
|
||||
## 3. Technical Background and Existing Technical Solutions |
||||
|
||||
### 3.1 Broad Technical Background |
||||
|
||||
Traditional programming languages can generally be divided into two categories. |
||||
|
||||
The first category is statically compiled languages, such as C, C++, Go, and Rust. They complete compilation before running, and types, function signatures, and memory layouts are mostly determined at compile time, resulting in high runtime performance. However, such languages generally have weaker development flexibility, with limited support for dynamic loading, dynamic method calls, and runtime modification of data structures. |
||||
|
||||
The second category is dynamically interpreted languages, such as PHP, Python, Ruby, and JavaScript. They resolve variable types, function calls, object properties, and method dispatch at runtime, resulting in high development efficiency and flexible expressiveness. However, such languages generally require virtual machines, interpreters, dynamic type determination, and runtime lookups, and their performance is inferior to statically compiled languages. |
||||
|
||||
Existing JIT technologies attempt to compile hot code into machine code at runtime, but JIT still relies on runtime sampling, type feedback, and hot-spot detection, and it is difficult to obtain fully deterministic native code before deployment. Traditional AOT technologies tend to fully staticize a program, but they easily lose language semantics when facing dynamic-language features such as variable functions, dynamic methods, magic methods, reflection, dynamic properties, closures, and callbacks. |
||||
|
||||
### 3.2 Narrow Technical Background |
||||
|
||||
Swoole-Compiler targets PHP programs. PHP programs have the following dynamic characteristics: |
||||
|
||||
- Variables can hold any type; |
||||
- Functions and methods can be called dynamically through strings or arrays; |
||||
- Objects can handle dynamic behavior through magic methods such as `__call()`, `__get()`, `__set()`; |
||||
- Namespaces, use aliases, traits, inheritance, and method overrides affect the actual call target; |
||||
- Parameters support default values, named arguments, variadic parameters, and reference parameters; |
||||
- PHP built-in functions and user functions can be called interchangeably; |
||||
- A program can run either as a PHP extension or as a standalone binary program. |
||||
|
||||
If the compiler forcibly converts all PHP code into static C++ calls, it will break the dynamic semantics described above. If dynamic interpreted execution is fully preserved, the performance advantages of AOT cannot be obtained. |
||||
|
||||
Therefore, a new hybrid-state programming language design is needed: enter static state in regions that can be statically determined, enter dynamic state in regions that cannot, and achieve interoperation between the two states through a unified boundary mechanism. |
||||
|
||||
### 3.3 Closest Existing Technical Solutions |
||||
|
||||
Existing technologies include: |
||||
|
||||
1. **Pure interpreted execution** |
||||
PHP source code is interpreted and executed by the Zend VM. This solution has strong compatibility, but every function call, property access, and array operation relies on runtime dynamic mechanisms. |
||||
|
||||
2. **Pure static compilation** |
||||
The entire source program is converted to C/C++ or machine code. This solution has high performance, but has difficulty supporting dynamic language features, usually requiring significant syntax restrictions or changes to language semantics. |
||||
|
||||
3. **JIT compilation** |
||||
Compilation is performed at runtime based on hot paths and type feedback. This solution can improve hot-spot performance, but still requires interpreter cooperation, and the compilation result depends on runtime state. |
||||
|
||||
4. **Handwritten extensions or FFI** |
||||
Dynamic languages call static code through C/C++ extensions or foreign function interfaces. This solution can partially improve performance, but lacks a unified language-level compilation model between static and dynamic code. |
||||
|
||||
## 4. Drawbacks of Existing Technologies and Objectives of the Present Invention |
||||
|
||||
### 4.1 Drawbacks of Existing Technologies |
||||
|
||||
Existing solutions have the following problems: |
||||
|
||||
1. Pure interpreted execution cannot fully exploit static types and native code performance. |
||||
2. Pure static compilation has difficulty being compatible with dynamic language features such as dynamic calls, magic methods, dynamic properties, and reflection. |
||||
3. JIT relies on runtime hot spots and type feedback, making it difficult to obtain stable, predictable compilation artifacts before deployment. |
||||
4. Handwritten extensions require developers to explicitly maintain interfaces, type conversions, and lifetimes between dynamic and static languages. |
||||
5. Function, class, property, and method lookups in dynamic languages usually rely on strings, and repeated lookups are costly. |
||||
6. Traditional compilers often use "whether it can be fully staticized" as the criterion, lacking a language-level model expressing the coexistence of multiple execution states within the same program. |
||||
|
||||
### 4.2 Objectives of the Present Invention |
||||
|
||||
The objective of the present invention is to provide a method and system for compilation and execution of a hybrid-state programming language, so that the same dynamic language program can be divided into static state and dynamic state: |
||||
|
||||
- In static state, the compiler generates C++ direct calls, strongly typed variables, strongly typed containers, and native code; |
||||
- In dynamic state, the system preserves the PHP VM's dynamic lookup, dynamic calls, dynamic properties, callbacks, and general zval semantics; |
||||
- Between the two states, interoperation is achieved through ArgInfo, wrapper functions, symbol caching, type conversion, and dynamic fallback mechanisms. |
||||
|
||||
In this way, the performance advantages of a static language and the flexibility advantages of a dynamic language are both obtained. |
||||
|
||||
## 5. Technical Solution of the Present Invention |
||||
|
||||
### 5.1 Hybrid-State Language Model |
||||
|
||||
The present invention proposes a "hybrid-state" language model. This model does not simply compile a dynamic language into a static language; instead, with the joint support of the compiler and the runtime, it divides statements, expressions, functions, methods, variables, and calls in a program into different states. |
||||
|
||||
The hybrid state includes at least: |
||||
|
||||
1. **Static function state**: function definitions, parameter types, and return types can be determined, generating C++ Native functions. |
||||
2. **Static object state**: the class of an object variable can be determined, and method calls can be directly linked to C++ functions. |
||||
3. **Static data state**: variables can be mapped to C++ native types or C++ template containers. |
||||
4. **Dynamic value state**: variables use `php::Var`, zval, or PHP objects for storage, preserving dynamic type semantics. |
||||
5. **Dynamic call state**: when the function name, method name, or object type cannot be determined, calls are made through the PHP runtime. |
||||
6. **Boundary bridging state**: between static and dynamic states, argument extraction, type conversion, return value write-back, and symbol caching are performed. |
||||
|
||||
### 5.2 System Composition |
||||
|
||||
```text |
||||
Figure 1: Hybrid-state programming language system composition diagram |
||||
|
||||
PHP source code / project configuration |
||||
| |
||||
v |
||||
Preprocessing and symbol extraction module |
||||
| |
||||
v |
||||
Dependency ordering and semantic analysis module |
||||
| |
||||
v |
||||
Hybrid-state determination module |
||||
| |
||||
+--> Static-state code generation module |
||||
| | |
||||
| +--> C++ function direct link |
||||
| +--> C++ native types |
||||
| +--> C++ template containers |
||||
| |
||||
+--> Dynamic-state code generation module |
||||
| | |
||||
| +--> PHP VM dynamic calls |
||||
| +--> zval / php::Var dynamic values |
||||
| +--> magic methods and callbacks |
||||
| |
||||
+--> Boundary bridging module |
||||
| |
||||
+--> ArgInfo argument conversion |
||||
+--> Zend wrapper functions |
||||
+--> symbol caching |
||||
+--> dynamic fallback |
||||
| |
||||
v |
||||
C++ source code and extension registration code |
||||
| |
||||
v |
||||
PHP extension or executable binary |
||||
``` |
||||
|
||||
### 5.3 Method Flow |
||||
|
||||
```text |
||||
Figure 2: Hybrid-state compilation and execution flow diagram |
||||
|
||||
Step S1: Read PHP source files and project configuration; |
||||
Step S2: Parse the abstract syntax tree; |
||||
Step S3: Extract symbols such as functions, classes, interfaces, traits, constants, properties, and methods; |
||||
Step S4: Order files by dependency based on symbol usage relationships; |
||||
Step S5: Resolve the ArgInfo of functions and methods; |
||||
Step S6: Determine for each expression or call site whether it can enter static state; |
||||
Step S7: If the target, types, and arguments can be determined, generate C++ static direct link code; |
||||
Step S8: If there is a risk of dynamic semantics, generate dynamic-state call code; |
||||
Step S9: Generate argument conversion, return value conversion, and wrapper functions for the boundary between static and dynamic states; |
||||
Step S10: Generate runtime cache mappings for functions, classes, methods, and properties; |
||||
Step S11: Compile the C++ code to produce a PHP extension or executable program; |
||||
Step S12: At runtime, execute the static direct link or dynamic fallback path according to the generated code. |
||||
``` |
||||
|
||||
### 5.4 Preprocessing and Symbol Extraction |
||||
|
||||
Before formally generating C++ code, the compiler first scans the PHP source code and extracts: |
||||
|
||||
- Function definitions; |
||||
- Class definitions; |
||||
- Interface definitions; |
||||
- Trait definitions; |
||||
- Class properties; |
||||
- Class methods; |
||||
- Class constants; |
||||
- Global constants; |
||||
- Namespaces and use aliases; |
||||
- Usage relationships among functions, classes, methods, and constants. |
||||
|
||||
It then builds a symbol dependency graph and performs topological sorting on the source files. This improves the success rate of static analysis and avoids functions or classes being temporarily invisible due to file ordering. |
||||
|
||||
### 5.5 ArgInfo Unified Parameter Model |
||||
|
||||
The present invention uses ArgInfo to describe function and method parameters, including: |
||||
|
||||
```text |
||||
Parameter name; |
||||
Parameter type; |
||||
Object class name; |
||||
Whether it is a reference parameter; |
||||
Whether it is a variadic parameter; |
||||
Whether it is nullable; |
||||
Default value; |
||||
Whether it is an UnsafePtr; |
||||
Whether it is a constructor property promotion parameter. |
||||
``` |
||||
|
||||
ArgInfo serves two directions simultaneously: |
||||
|
||||
1. **AOT internal direct-link calls**: the compiler reorders named arguments, fills default values, merges variadic parameters, and generates C++ direct calls based on ArgInfo. |
||||
2. **Zend runtime entry calls**: when a compiled function is called by the PHP VM as a PHP extension function, the wrapper function reads parameters from the call stack and converts types according to ArgInfo. |
||||
|
||||
This design allows static and dynamic states to share the same set of parameter semantics. |
||||
|
||||
### 5.6 Static-State Code Generation |
||||
|
||||
When the compiler can determine the call target, it generates a C++ direct call. |
||||
|
||||
Example: |
||||
|
||||
```php |
||||
function add(int $a, int $b): int |
||||
{ |
||||
return $a + $b; |
||||
} |
||||
|
||||
add(1, 2); |
||||
``` |
||||
|
||||
Generates something like: |
||||
|
||||
```cpp |
||||
php_add(php::toInt(1L), php::toInt(2L)); |
||||
``` |
||||
|
||||
For object methods: |
||||
|
||||
```php |
||||
$obj->run($arg); |
||||
``` |
||||
|
||||
If the class of `$obj` can be determined, and there is no method override or dynamic magic method risk, it generates: |
||||
|
||||
```cpp |
||||
php_Class_run(obj, converted_arg); |
||||
``` |
||||
|
||||
Static state also includes: |
||||
|
||||
- Native types such as `int`, `float`, `bool`; |
||||
- C++ template containers such as `std::array`, `std::vector`; |
||||
- Property access on objects whose class can be determined; |
||||
- Default argument and variadic parameter handling for functions that can be determined; |
||||
- Constant and class constant access that can be determined at compile time. |
||||
|
||||
### 5.7 Dynamic-State Code Generation |
||||
|
||||
The compiler enters dynamic state in the following cases: |
||||
|
||||
- The function name comes from a variable; |
||||
- The method name comes from a variable; |
||||
- The actual type of an object cannot be determined; |
||||
- The class may have a `__call()` magic method; |
||||
- A method is overridden by a subclass, and static direct link may change dynamic dispatch semantics; |
||||
- A PHP built-in dynamic function needs to be called; |
||||
- Callbacks, closures, or placeholder expressions cannot be fully determined statically; |
||||
- Variables use general `mixed` or dynamic array semantics. |
||||
|
||||
Dynamic state generates something like: |
||||
|
||||
```cpp |
||||
php::call(function_ptr, arg_list); |
||||
``` |
||||
|
||||
or: |
||||
|
||||
```cpp |
||||
object.call(method_ptr, arg_list); |
||||
``` |
||||
|
||||
Dynamic state preserves the PHP VM's function lookup, method dispatch, zval types, and dynamic callback semantics. |
||||
|
||||
### 5.8 Static Direct Link vs. Dynamic Fallback Determination |
||||
|
||||
The present invention does not require all code to be staticized; instead, a determination is made at each call site. |
||||
|
||||
```text |
||||
Figure 3: Call-site state determination flow |
||||
|
||||
Call expression |
||||
| |
||||
v |
||||
Is the function name or method name a literal? |
||||
| |
||||
+-- No --> dynamic-state call |
||||
| |
||||
+-- Yes |
||||
| |
||||
v |
||||
Can a Native target be found in the symbol table? |
||||
| |
||||
+-- No --> dynamic-state call |
||||
| |
||||
+-- Yes |
||||
| |
||||
v |
||||
Can the object class or function signature be determined? |
||||
| |
||||
+-- No --> dynamic-state call |
||||
| |
||||
+-- Yes |
||||
| |
||||
v |
||||
Are there risks such as magic methods, overrides, or dynamic callbacks? |
||||
| |
||||
+-- Yes --> dynamic-state call |
||||
| |
||||
+-- No --> static-state direct link |
||||
``` |
||||
|
||||
This mechanism means the language runtime is not a single state, but automatically selects the best state by code location. |
||||
|
||||
### 5.9 Boundary Bridging Mechanism |
||||
|
||||
Static and dynamic states interoperate through a boundary bridging mechanism. |
||||
|
||||
#### 5.9.1 Dynamic Call to Static Function |
||||
|
||||
When the PHP VM calls a compiled function, it enters a Zend wrapper function: |
||||
|
||||
```text |
||||
Zend call entry point |
||||
| |
||||
v |
||||
Read call stack arguments |
||||
| |
||||
v |
||||
Convert arguments according to ArgInfo |
||||
| |
||||
v |
||||
Call the Native function |
||||
| |
||||
v |
||||
Write the return value back to return_value |
||||
``` |
||||
|
||||
#### 5.9.2 Static Code Calling Dynamic Function |
||||
|
||||
When AOT code cannot determine the target, it generates a dynamic call: |
||||
|
||||
```text |
||||
C++ static code |
||||
| |
||||
v |
||||
Construct a PHP dynamic argument list |
||||
| |
||||
v |
||||
Look up the function or method |
||||
| |
||||
v |
||||
Call the PHP VM dynamic path |
||||
| |
||||
v |
||||
Return a php::Var dynamic value |
||||
``` |
||||
|
||||
#### 5.9.3 Static Data Flowing into Dynamic State |
||||
|
||||
For example, when a strongly typed container is assigned to a normal PHP variable, it is automatically converted to a PHP Array: |
||||
|
||||
```php |
||||
$arr = $vector; |
||||
``` |
||||
|
||||
Generates: |
||||
|
||||
```cpp |
||||
arr = php::toArray(vector); |
||||
``` |
||||
|
||||
#### 5.9.4 Dynamic Object Flowing into Static State |
||||
|
||||
When an object is obtained from a dynamic array or function return value, and a subsequent static method call is needed, the compiler can be informed of the object's class name through a compile-time type declaration function or a type conversion function, thereby entering static object state. |
||||
|
||||
### 5.10 Symbol Caching Mechanism |
||||
|
||||
To reduce the runtime string lookup overhead of dynamic state, the present invention establishes integer IDs and cache arrays for symbols: |
||||
|
||||
```cpp |
||||
zend_class_entry *class_map[N]; |
||||
zend_function *func_map[M]; |
||||
uint32_t property_map[K]; |
||||
``` |
||||
|
||||
On first lookup, the Zend structure pointer is obtained by string name and cached; subsequent calls access it quickly through the integer ID. |
||||
|
||||
Example: |
||||
|
||||
```cpp |
||||
zend_function *get_func(int id, const php::Str &name) { |
||||
if (func_map[id] == nullptr) { |
||||
func_map[id] = php::getFunction(name); |
||||
} |
||||
return func_map[id]; |
||||
} |
||||
``` |
||||
|
||||
This mechanism makes dynamic state still more efficient than the pure interpreted path. |
||||
|
||||
### 5.11 Dual Runtime Forms |
||||
|
||||
The present invention supports two runtime forms: |
||||
|
||||
1. **Extension form**: the compilation result is a PHP extension that can be loaded into PHP-FPM or CLI and called by the PHP VM. |
||||
2. **Binary form**: the compilation result is an executable program, entered through the `main()` function and directly runnable. |
||||
|
||||
These two forms share the same hybrid-state compilation model. |
||||
|
||||
## 6. Key Points and Points to Protect |
||||
|
||||
1. Propose a hybrid-state programming language model that allows static state and dynamic state to coexist in the same dynamic language program. |
||||
2. Automatically determine static or dynamic state in the compiler by function, method, variable, expression, and call site. |
||||
3. Generate C++ static direct link code for determinable targets, and automatically fall back to PHP dynamic call code for undeterminable targets. |
||||
4. Use ArgInfo as the parameter semantic model shared by static and dynamic states. |
||||
5. Generate Zend wrapper functions so the dynamic runtime can call AOT-generated Native functions. |
||||
6. Construct dynamic argument lists in AOT static code so static code can call PHP dynamic functions, dynamic methods, or callbacks. |
||||
7. Use a symbol caching mechanism to reduce the string lookup overhead of functions, classes, methods, and properties in dynamic-state runtime. |
||||
8. Support two runtime artifacts, extension form and binary form, both based on the same hybrid-state language model. |
||||
9. Support native types and C++ template containers in static state, and preserve zval, php::Var, PHP Array, and PHP object semantics in dynamic state. |
||||
|
||||
## 7. Advantages over Existing Technologies |
||||
|
||||
Compared with traditional statically compiled languages, the present invention preserves the dynamic call, dynamic type, dynamic object, and callback capabilities of dynamic languages, without requiring full program staticization. |
||||
|
||||
Compared with traditional dynamically interpreted languages, the present invention can generate C++ direct calls, native types, and strongly typed containers in regions determinable at compile time, significantly reducing runtime lookup and dynamic type overhead. |
||||
|
||||
Compared with JIT, the present invention generates stable compilation artifacts before deployment, does not rely on runtime hot-spot sampling and type feedback, and is suitable for production environments that require stable deployment, source code protection, and performance improvement. |
||||
|
||||
Compared with handwritten extensions, the present invention automatically generates wrapper functions, argument conversion, symbol caching, and dynamic fallback paths through the compiler, reducing the cost of manually bridging between PHP and C++. |
||||
|
||||
## 8. Alternative Embodiments |
||||
|
||||
The present invention is not limited to PHP; it can also be applied to dynamic languages such as Python, JavaScript, Ruby, and Lua. The target static language is not limited to C++; it can also be Rust, Go, C, or LLVM intermediate representation. |
||||
|
||||
The granularity of hybrid-state determination can be function-level, basic-block-level, statement-level, or expression-level. Symbol caching can also be implemented using hash tables, arrays, handle tables, or runtime inline caching. |
||||
|
||||
## 9. Confidentiality Statement |
||||
|
||||
This document involves the overall compilation model of Swoole-Compiler, the static direct link and dynamic fallback strategy, the ArgInfo boundary bridging mechanism, and the symbol caching solution. Before formal application, it is recommended to manage it as internal technical material. |
||||
@ -0,0 +1,355 @@ |
||||
# Rebuilding the PHPX WASM Static Library |
||||
|
||||
This document is aimed at TypePHP/PHPX developers and explains how to recompile and install the |
||||
PHPX static library for `wasm32-wasip2`. Ordinary TypePHP users do not need to perform these steps; release packages should provide the complete |
||||
WASI SDK directly. |
||||
|
||||
## Directory Convention |
||||
|
||||
This document assumes the source layout is as follows: |
||||
|
||||
```text |
||||
/home/swoole/workspace/aot/ |
||||
├── compiler/ |
||||
└── phpx/ |
||||
``` |
||||
|
||||
It is recommended to set the PHPX root directory first: |
||||
|
||||
```shell |
||||
export PHPX_HOME=/home/swoole/workspace/aot/phpx |
||||
``` |
||||
|
||||
The installation prefix is fixed to: |
||||
|
||||
```text |
||||
$PHPX_HOME/wasm/wasm32-wasip2 |
||||
``` |
||||
|
||||
This directory is both the input of the existing PHP/WASI SDK and the installation location of the PHPX build results: |
||||
|
||||
```text |
||||
wasm/wasm32-wasip2/ |
||||
├── include/php/ PHP/WASI headers |
||||
├── include/phpx/ PHPX/TypePHP runtime headers |
||||
├── lib/libphp.a |
||||
├── lib/libphpx.a |
||||
├── lib/libgmp.a |
||||
├── lib/libgmpxx.a |
||||
├── lib/libmpfr.a |
||||
├── lib/libmpdec.a |
||||
├── lib/libmpdec++.a |
||||
└── .typephp-wasi-sdk-abi |
||||
``` |
||||
|
||||
Do not copy host-platform `libphpx.so`, `phpx.dll`, or `.a` files here. |
||||
WASM static libraries contain the target ABI and cannot be used across WASI, Linux, macOS, or Windows. |
||||
|
||||
## Toolchain Preparation |
||||
|
||||
PHPX WASM currently supports only WASI 0.2 Preview 2. Add the WASI SDK to `PATH`: |
||||
|
||||
```shell |
||||
export PATH=/opt/wasi-sdk-33.0/bin:$PATH |
||||
``` |
||||
|
||||
`PATH` only lets the shell and build tools find the WASI SDK programs; it does not make CMake automatically select the |
||||
WASI target. When configuring the build directory for the first time, you must still pass |
||||
`-DCMAKE_TOOLCHAIN_FILE=.../wasi-sdk-p2.cmake`. If you omit it, CMake will choose the host platform's |
||||
`/usr/bin/cc` and `/usr/bin/c++`, and PHPX's target check will immediately reject that configuration. |
||||
|
||||
Confirm the necessary tools: |
||||
|
||||
```shell |
||||
command -v wasm32-wasip2-clang |
||||
command -v wasm32-wasip2-clang++ |
||||
command -v llvm-ar |
||||
command -v llvm-ranlib |
||||
command -v llvm-nm |
||||
command -v cmake |
||||
command -v ninja |
||||
``` |
||||
|
||||
Confirm the compile target: |
||||
|
||||
```shell |
||||
wasm32-wasip2-clang++ --print-target-triple |
||||
``` |
||||
|
||||
It must output: |
||||
|
||||
```text |
||||
wasm32-unknown-wasip2 |
||||
``` |
||||
|
||||
The installation prefix must already contain PHP/WASI headers and `libphp.a` matching the current PHPX: |
||||
|
||||
```shell |
||||
test -f "$PHPX_HOME/wasm/wasm32-wasip2/include/php/main/php.h" |
||||
test -f "$PHPX_HOME/wasm/wasm32-wasip2/lib/libphp.a" |
||||
``` |
||||
|
||||
## Daily Development: Rebuild PHPX Directly with CMake |
||||
|
||||
When a PHPX `.cc` or header file changes, incrementally rebuild directly using `phpx/wasm/CMakeLists.txt`. |
||||
This is the recommended flow for daily development; it does not re-download or recompile PHP, GMP, or MPFR, nor does it regenerate |
||||
`libphp.a`. |
||||
|
||||
First, locate the CMake toolchain from the current WASI compiler to avoid depending on a hardcoded SDK version path: |
||||
|
||||
```shell |
||||
WASI_RESOURCE_DIR="$(wasm32-wasip2-clang++ --print-resource-dir)" |
||||
WASI_SDK_ROOT="$(cd "$WASI_RESOURCE_DIR/../../.." && pwd)" |
||||
WASI_CMAKE_TOOLCHAIN="$WASI_SDK_ROOT/share/cmake/wasi-sdk-p2.cmake" |
||||
test -f "$WASI_CMAKE_TOOLCHAIN" |
||||
``` |
||||
|
||||
### Using Ninja (Recommended) |
||||
|
||||
First configure a persistent build directory: |
||||
|
||||
```shell |
||||
cmake \ |
||||
-S "$PHPX_HOME/wasm" \ |
||||
-B "$PHPX_HOME/build/wasm32-wasip2" \ |
||||
-G Ninja \ |
||||
-DCMAKE_TOOLCHAIN_FILE="$WASI_CMAKE_TOOLCHAIN" \ |
||||
-DCMAKE_BUILD_TYPE=Release \ |
||||
-DPHPX_WASI_SDK_DIR="$PHPX_HOME/wasm/wasm32-wasip2" \ |
||||
-DCMAKE_INSTALL_PREFIX="$PHPX_HOME/wasm/wasm32-wasip2" |
||||
``` |
||||
|
||||
The toolchain takes effect when CMake executes `project()`, so it can only be set during the first configuration of a build directory. |
||||
If the directory was configured without a toolchain before and has already cached the host compiler, do not append arguments onto the existing cache; |
||||
use a new build directory instead, for example: |
||||
|
||||
```shell |
||||
cmake \ |
||||
-S "$PHPX_HOME/wasm" \ |
||||
-B "$PHPX_HOME/build/wasm32-wasip2-wasi" \ |
||||
-G Ninja \ |
||||
-DCMAKE_TOOLCHAIN_FILE="$WASI_CMAKE_TOOLCHAIN" \ |
||||
-DCMAKE_BUILD_TYPE=Release \ |
||||
-DPHPX_WASI_SDK_DIR="$PHPX_HOME/wasm/wasm32-wasip2" \ |
||||
-DCMAKE_INSTALL_PREFIX="$PHPX_HOME/wasm/wasm32-wasip2" |
||||
``` |
||||
|
||||
Subsequent build/install commands should also use this new directory. |
||||
|
||||
Compile and install: |
||||
|
||||
```shell |
||||
cmake --build "$PHPX_HOME/build/wasm32-wasip2" --parallel 16 |
||||
cmake --install "$PHPX_HOME/build/wasm32-wasip2" |
||||
``` |
||||
|
||||
When the PHPX source changes again later, you only need to run: |
||||
|
||||
```shell |
||||
cmake --build "$PHPX_HOME/build/wasm32-wasip2" --parallel 16 |
||||
cmake --install "$PHPX_HOME/build/wasm32-wasip2" |
||||
``` |
||||
|
||||
CMake/Ninja only recompiles the changed source files and then updates `libphpx.a` in the installation directory. |
||||
|
||||
### Using Make |
||||
|
||||
`make` can be used, but the `Unix Makefiles` generator must be selected during the first configuration, using a different build |
||||
directory; you cannot switch generators in a directory already configured by Ninja: |
||||
|
||||
```shell |
||||
cmake \ |
||||
-S "$PHPX_HOME/wasm" \ |
||||
-B "$PHPX_HOME/build/wasm32-wasip2-make" \ |
||||
-G "Unix Makefiles" \ |
||||
-DCMAKE_TOOLCHAIN_FILE="$WASI_CMAKE_TOOLCHAIN" \ |
||||
-DCMAKE_BUILD_TYPE=Release \ |
||||
-DPHPX_WASI_SDK_DIR="$PHPX_HOME/wasm/wasm32-wasip2" \ |
||||
-DCMAKE_INSTALL_PREFIX="$PHPX_HOME/wasm/wasm32-wasip2" |
||||
|
||||
make -C "$PHPX_HOME/build/wasm32-wasip2-make" -j16 |
||||
make -C "$PHPX_HOME/build/wasm32-wasip2-make" install |
||||
``` |
||||
|
||||
After later modifying PHPX code, just repeat the two `make` commands. You can also use the generator-independent form: |
||||
|
||||
```shell |
||||
cmake --build "$PHPX_HOME/build/wasm32-wasip2-make" --parallel 16 |
||||
cmake --install "$PHPX_HOME/build/wasm32-wasip2-make" |
||||
``` |
||||
|
||||
The artifacts of Ninja and Make are the same; Ninja is generally faster in dependency scanning and incremental builds, so internal development defaults |
||||
to Ninja. |
||||
|
||||
This flow updates: |
||||
|
||||
- `lib/libphpx.a` |
||||
- `lib/libmpdec.a` and `lib/libmpdec++.a` (recompiled only when the related source changes) |
||||
- PHPX public headers under `include/phpx/` |
||||
- `.typephp-wasi-runtime-abi` |
||||
|
||||
It does not update `libphp.a`, GMP, or MPFR, nor does it rewrite the full SDK's |
||||
`.typephp-wasi-sdk-abi`. Therefore, this flow should be run on an already fully installed SDK. |
||||
|
||||
### Force Recompiling PHPX |
||||
|
||||
When you suspect that old objects or the CMake cache are no longer trustworthy, prefer using a new, explicit build directory: |
||||
|
||||
```shell |
||||
cmake \ |
||||
-S "$PHPX_HOME/wasm" \ |
||||
-B "$PHPX_HOME/build/wasm32-wasip2-clean" \ |
||||
-G Ninja \ |
||||
-DCMAKE_TOOLCHAIN_FILE="$WASI_CMAKE_TOOLCHAIN" \ |
||||
-DCMAKE_BUILD_TYPE=Release \ |
||||
-DPHPX_WASI_SDK_DIR="$PHPX_HOME/wasm/wasm32-wasip2" \ |
||||
-DCMAKE_INSTALL_PREFIX="$PHPX_HOME/wasm/wasm32-wasip2" |
||||
|
||||
cmake --build "$PHPX_HOME/build/wasm32-wasip2-clean" --parallel 16 |
||||
cmake --install "$PHPX_HOME/build/wasm32-wasip2-clean" |
||||
``` |
||||
|
||||
This does not delete `libphp.a` and the dependency libraries in the installation directory, nor does it mix in old CMake configuration. |
||||
|
||||
## First Build or Rebuilding PHPX Numeric Dependencies |
||||
|
||||
Use PHPX's unified build entry in the following cases: |
||||
|
||||
- Setting up the PHPX WASI installation directory for the first time; |
||||
- GMP or MPFR version, patch, or compile parameter changes; |
||||
- Changes to PHPX vendored mpdecimal or its WASI configuration; |
||||
- The need to check and install all PHPX WASI headers and static libraries at once. |
||||
|
||||
```shell |
||||
cd "$PHPX_HOME" |
||||
|
||||
./wasm/build.sh \ |
||||
--prefix "$PHPX_HOME/wasm/wasm32-wasip2" \ |
||||
--build-dir "$PHPX_HOME/build/wasm32-wasip2-sdk" \ |
||||
--jobs 16 |
||||
``` |
||||
|
||||
Explicitly use `$PHPX_HOME/build/` to avoid the default `/tmp` build directory being lost after a reboot. The downloaded GMP and |
||||
MPFR source and build cache are retained and can be reused in subsequent builds. |
||||
|
||||
This entry builds or installs: |
||||
|
||||
- `libphpx.a` |
||||
- `libgmp.a`, `libgmpxx.a` |
||||
- `libmpfr.a` |
||||
- `libmpdec.a`, `libmpdec++.a` |
||||
- The corresponding headers and the PHPX runtime ABI marker |
||||
|
||||
It requires PHP/WASI headers to already exist in the installation prefix; it does not build `libphp.a`. |
||||
|
||||
## PHP ABI Changes: Rebuilding the Full SDK |
||||
|
||||
If the PHP source, extension set, PHP configuration, Zend ABI, or PHP installed headers change, you must rebuild the full SDK from the |
||||
TypePHP compiler repository, and you cannot replace only `libphpx.a`: |
||||
|
||||
```shell |
||||
cd /home/swoole/workspace/aot/compiler |
||||
|
||||
./wasm/build-sdk.sh \ |
||||
--prefix "$PHPX_HOME/wasm/wasm32-wasip2" \ |
||||
--php-source "$PWD/projects/php-8.5.9" \ |
||||
--phpx-source "$PHPX_HOME" \ |
||||
--build-dir "$PWD/build/wasm-sdk" \ |
||||
--jobs 16 |
||||
``` |
||||
|
||||
The full build installs the PHP and PHPX parts in sequence, and writes the following after all artifacts are verified: |
||||
|
||||
```text |
||||
.typephp-wasi-sdk-abi |
||||
``` |
||||
|
||||
Do not forge this marker by hand. The existence of the marker only means the build flow declares ABI compatibility; it cannot fix actually mixed |
||||
old headers or static libraries. |
||||
|
||||
## Artifact Verification |
||||
|
||||
After installation completes, check the key files: |
||||
|
||||
```shell |
||||
WASI_PREFIX="$PHPX_HOME/wasm/wasm32-wasip2" |
||||
|
||||
test -s "$WASI_PREFIX/lib/libphpx.a" |
||||
test -s "$WASI_PREFIX/lib/libphp.a" |
||||
test -f "$WASI_PREFIX/include/phpx/phpx.h" |
||||
test -f "$WASI_PREFIX/include/phpx/phpx_helper.h" |
||||
test -f "$WASI_PREFIX/include/phpx/typephp_helper.h" |
||||
|
||||
llvm-ar t "$WASI_PREFIX/lib/libphpx.a" | head |
||||
cat "$WASI_PREFIX/.typephp-wasi-runtime-abi" |
||||
cat "$WASI_PREFIX/.typephp-wasi-sdk-abi" |
||||
``` |
||||
|
||||
The current markers should be: |
||||
|
||||
```text |
||||
typephp-wasip2-phpx-abi-v1 |
||||
typephp-wasip2-sdk-abi-v4 |
||||
``` |
||||
|
||||
The marker versions will be upgraded as the ABI design evolves; if the expected values in the code have changed, follow the current build scripts |
||||
rather than writing old values back just to pass detection. |
||||
|
||||
## TypePHP Regression Verification |
||||
|
||||
First verify the Wasmtime component: |
||||
|
||||
```shell |
||||
cd /home/swoole/workspace/aot/compiler |
||||
|
||||
PHPX_HOME="$PHPX_HOME" \ |
||||
./run-tests.php --wasm --compiler ./bin/tpc.php tests/wasm/ |
||||
``` |
||||
|
||||
Then verify that Wasmtime and Chrome output are consistent, and cover parallel build/output directory isolation: |
||||
|
||||
```shell |
||||
PHPX_HOME="$PHPX_HOME" \ |
||||
./run-tests.php -j 4 --target wasm-all --compiler ./bin/tpc.php tests/wasm/ |
||||
``` |
||||
|
||||
Browser tests also require `jco`, Node.js, and Chrome to be in `PATH`. `wasm-all` runs each case in |
||||
Wasmtime and Chrome separately, and compares the output of both sides. |
||||
|
||||
Finally build the browser example: |
||||
|
||||
```shell |
||||
cd examples/wasm-hello |
||||
PHPX_HOME="$PHPX_HOME" ../../bin/tpc.php project.yml |
||||
npm run build |
||||
``` |
||||
|
||||
## Common Errors |
||||
|
||||
### `PersistentCacheSlot` or PHPX helpers are undefined |
||||
|
||||
The generated code uses a new PHPX header/API, but `include/phpx/` or |
||||
`lib/libphpx.a` in the installation prefix is still an old version. Run the "Daily development: rebuild PHPX only" flow, and make sure the configuration and installation |
||||
use the same `PHPX_WASI_SDK_DIR`/`CMAKE_INSTALL_PREFIX`. |
||||
|
||||
### `TypePHP WASI SDK is missing or ABI-incompatible` |
||||
|
||||
Check whether `PHPX_HOME` points to the actual PHPX root directory, and whether the full SDK marker, PHP/PHPX headers, |
||||
and static libraries come from the same compatible build. Run the full SDK rebuild when the PHP ABI has changed. |
||||
|
||||
### CMake detects the host compiler |
||||
|
||||
You must pass the WASI SDK's `wasi-sdk-p2.cmake`. Do not use the host |
||||
`CMakeLists.txt` in the PHPX root directory to build WASM directly. Adding the WASI SDK to `PATH` is not equivalent to loading the CMake |
||||
toolchain. If `CMakeCache.txt` has already recorded `/usr/bin/cc` or `/usr/bin/c++`, use a |
||||
new build directory to reconfigure. |
||||
|
||||
### TypePHP still links the old implementation after modifying PHPX |
||||
|
||||
Confirm that `PHPX_HOME` takes priority over the Composer directory, and check the actual artifact time: |
||||
|
||||
```shell |
||||
stat "$PHPX_HOME/wasm/wasm32-wasip2/lib/libphpx.a" |
||||
``` |
||||
|
||||
TypePHP should read both headers and static libraries from the same `$PHPX_HOME/wasm/wasm32-wasip2`. |
||||
@ -0,0 +1,223 @@ |
||||
# PHP 8.4 Property Hook Integration Design |
||||
|
||||
This document records how the TypePHP compiler and PHPX implement PHP 8.4 Property Hooks, focusing on Zend metadata registration, object introspection, memory lifetime, and version compatibility boundaries. This is an internal maintenance document; user-facing syntax documentation belongs in the external documentation repository. |
||||
|
||||
A Property Hook without an implementation body in an Interface is an abstract property contract and does not go through the concrete-class lowering flow described here. For its model, variance checking, and Zend metadata registration, see [Interface Property Hook Implementation Plan](INTERFACE_PROPERTY_HOOKS.md). |
||||
|
||||
## 1. Background |
||||
|
||||
TypePHP compiles Property Hook bodies into hidden AOT getters/setters. Doing only this step satisfies property reads/writes that the compiler explicitly identifies, but ZendVM does not know that these hidden methods represent Property Hooks, so the following dynamic capabilities diverge from PHP 8.4: |
||||
|
||||
- `ReflectionProperty::hasHooks()`, `getHooks()`, and `isVirtual()`; |
||||
- `get_object_vars()`, `json_encode()`, and `var_export()`; |
||||
- `foreach` traversal of objects; |
||||
- the storage difference between backed properties and virtual properties; |
||||
- dynamic property reads/writes initiated by ZendVM. |
||||
|
||||
TypePHP does not emulate these PHP behaviors separately. The compiler preserves Hook metadata after lowering, and PHPX wires the AOT methods into the native PHP 8.4 Property Hook structures when the class is registered during MINIT. After that, Reflection and object introspection reuse the standard ZendVM implementation. |
||||
|
||||
## 2. Compilation Flow |
||||
|
||||
### 2.1 AST lowering |
||||
|
||||
`PropertyHookLowering` converts each Hook into a hidden class method and records on the property AST: |
||||
|
||||
- the hidden method names corresponding to the getter/setter; |
||||
- whether the Hook accesses its own backing storage; |
||||
- whether the property is a virtual property. |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
public string $name { |
||||
get => strtoupper($this->name); |
||||
set => $this->name = trim($value); |
||||
} |
||||
``` |
||||
|
||||
This produces equivalent hidden getters/setters internally. `$this->name` inside the Hook is marked as backing access to avoid recursion by calling the Hook again. |
||||
|
||||
If the Hook does not access backing storage, the property is marked virtual. This conclusion must be obtained during the lowering stage, because the generated Zend property declaration needs it to decide whether to allocate a property slot. |
||||
|
||||
### 2.2 Class registration code |
||||
|
||||
After `gen_stub.php` declares the property and obtains the `zend_property_info *`, it generates: |
||||
|
||||
```cpp |
||||
typephp_register_property_hooks( |
||||
class_entry, |
||||
property_info, |
||||
getter_method_name, |
||||
setter_method_name |
||||
); |
||||
``` |
||||
|
||||
The call happens during the class's persistent registration stage, not on the request hot path. |
||||
|
||||
## 3. PHPX Registration Flow |
||||
|
||||
PHPX's `typephp_register_property_hooks()` is implemented only for PHP 8.4 and above, and lives in a TypePHP-specific helper. |
||||
|
||||
### 3.1 Locating the AOT implementation method |
||||
|
||||
PHPX finds the hidden method produced by lowering from the class method table: |
||||
|
||||
```cpp |
||||
zend_hash_str_find_ptr(&ce->function_table, method_name.data(), method_name.size()); |
||||
``` |
||||
|
||||
This method is an already-registered `zend_internal_function` whose handler ultimately enters the TypePHP-generated C++ getter/setter. The lookup happens only once; property reads/writes do not re-query the function table. |
||||
|
||||
### 3.2 Creating the Hook function descriptor |
||||
|
||||
The hidden function object in the class method table cannot be directly modified or reused. Zend Property Hooks require an independent function identity and property association: |
||||
|
||||
```cpp |
||||
hook->function_name = "$name::get"; // or "$name::set" |
||||
hook->prop_info = property_info; |
||||
``` |
||||
|
||||
PHPX therefore copies a `zend_internal_function` descriptor and replaces the Hook-specific fields. The copy does not produce a second C++ implementation; the handler, argument info, and other persistent data still come from the original AOT method. |
||||
|
||||
An independent function descriptor avoids breaking the class method table's key, reflection name, or ownership relationships when the hidden method is modified, and lets Reflection correctly report `$name::get` and `$name::set`. |
||||
|
||||
### 3.3 Mounting the property Hook |
||||
|
||||
PHP 8.4 added a Hook table to `zend_property_info`: |
||||
|
||||
```cpp |
||||
property_info->hooks[ZEND_PROPERTY_HOOK_GET] = getter; |
||||
property_info->hooks[ZEND_PROPERTY_HOOK_SET] = setter; |
||||
``` |
||||
|
||||
The following must also be updated: |
||||
|
||||
```cpp |
||||
ce->num_hooked_props++; |
||||
``` |
||||
|
||||
Zend's Reflection, object property construction, and inheritance checks all read this metadata. Registering only the hidden method without filling in `property_info->hooks` will not be recognized by Zend as a true Property Hook. |
||||
|
||||
### 3.4 Installing the Hook object iterator |
||||
|
||||
When the class has no custom iterator, PHPX sets: |
||||
|
||||
```cpp |
||||
ce->get_iterator = zend_hooked_object_get_iterator; |
||||
``` |
||||
|
||||
`zend_hooked_object_get_iterator()` is the `ZEND_API` exported by PHP 8.4 in `zend_property_hooks.h`. PHP itself installs this iterator when compiling classes that contain Property Hooks. |
||||
|
||||
The ordinary object iterator mainly traverses physical property slots, while the Hook iterator is also responsible for: |
||||
|
||||
- calling getters for backed and virtual properties; |
||||
- skipping virtual properties without a getter; |
||||
- enforcing property visibility rules; |
||||
- rejecting unsupported by-reference traversal; |
||||
- merging dynamic properties. |
||||
|
||||
Therefore PHPX should not duplicate a traversal implementation. Reusing Zend's exported implementation keeps `foreach` behavior consistent and reduces maintenance cost. |
||||
|
||||
## 4. Virtual property |
||||
|
||||
PHP 8.4 uses a special offset to represent a virtual property: |
||||
|
||||
```cpp |
||||
#define ZEND_VIRTUAL_PROPERTY_OFFSET ((uint32_t) -1) |
||||
``` |
||||
|
||||
When Zend declares a property, it needs `IS_UNDEF` as the declaration value to establish a virtual offset for a property with `ZEND_ACC_VIRTUAL`. Therefore the generated code uses: |
||||
|
||||
```cpp |
||||
zval default_value; |
||||
ZVAL_UNDEF(&default_value); |
||||
``` |
||||
|
||||
It must not be replaced with `null` or an ordinary default value, otherwise Zend may allocate a backing slot and `ReflectionProperty::isVirtual()` will return the wrong result. |
||||
|
||||
## 5. Object introspection and serialization |
||||
|
||||
When `ce->num_hooked_props` is nonzero, Zend's `zend_std_get_properties_for()` calls `zend_hooked_object_build_properties()` in scenarios such as JSON, `get_object_vars()`, and `var_export()`. That function reads the Hooked public property values. |
||||
|
||||
Serialization uses different semantics: |
||||
|
||||
- a virtual property has no persistent state and does not appear in the serialization result; |
||||
- a backed property serializes the backing value, not the value computed by the getter; |
||||
- private storage properties are still serialized according to PHP's property-name mangling rules. |
||||
|
||||
This difference is existing PHP 8.4 behavior and must not be overridden just to make JSON and serialization output identical. |
||||
|
||||
## 6. Lifetime and thread safety |
||||
|
||||
TypePHP AOT classes are registered as persistent internal classes. The Hook table, Hook function descriptors, and function names must share the same process-level lifetime, so PHPX uses: |
||||
|
||||
```cpp |
||||
pemalloc(size, true); |
||||
zend_string_init(data, length, true); |
||||
``` |
||||
|
||||
Request memory must not be used; otherwise the class entry would keep dangling pointers after RSHUTDOWN and the next request could crash when accessing properties or Reflection. |
||||
|
||||
Registration happens only in MINIT: |
||||
|
||||
- during request execution, Hook metadata is read-only; |
||||
- it does not need to be rebuilt on every request; |
||||
- it does not need to look up hidden methods on every property access; |
||||
- NTS has no locking overhead; |
||||
- under ZTS, registration completes before worker threads process requests, so the class entry is not concurrently modified. |
||||
|
||||
## 7. PHP version boundary |
||||
|
||||
The minimum version of both TypePHP and PHPX is PHP 8.4, so the Property Hook implementation directly uses the following PHP 8.4 ABI: |
||||
|
||||
- `zend_property_info::hooks`; |
||||
- `zend_class_entry::num_hooked_props`; |
||||
- `ZEND_PROPERTY_HOOK_*`; |
||||
- `ZEND_PROPERTY_HOOK_STRUCT_SIZE`; |
||||
- `ZEND_VIRTUAL_PROPERTY_OFFSET`; |
||||
- `zend_hooked_object_get_iterator()`. |
||||
|
||||
PHPX headers and CMake configuration reject headers/`php-config` below PHP 8.4. PHP 8.4 and 8.5 still build separate PHPX binaries; `--php-version` only controls source syntax and does not require exact minor-version parity with `libphp.so`, but both must be no lower than 8.4. |
||||
|
||||
## 8. ABI risk and upgrade checks |
||||
|
||||
`zend_hooked_object_get_iterator()` is an exported Zend API, but Property Hooks overall remain a version-dependent low-level Zend ABI. PHP 8.4 does not provide a complete high-level `zend_declare_property_hook()` extension API, so the current implementation must fill in Zend metadata. |
||||
|
||||
The rationale for this approach is: |
||||
|
||||
1. TypePHP and PHPX are version-locked and recompiled against specific PHP versions; |
||||
2. the registration flow matches the steps Zend's compiler performs for native Property Hooks; |
||||
3. only Zend's exported iterator is reused, without duplicating its complex implementation; |
||||
4. versions below PHP 8.4 are uniformly rejected at the build entry point; |
||||
5. all registration completes in MINIT, adding no name lookup to the request hot path. |
||||
|
||||
When upgrading the PHP version, the following must be checked: |
||||
|
||||
1. whether the Hook fields and ownership of `zend_property_info` changed; |
||||
2. whether `ZEND_PROPERTY_HOOK_COUNT` and Hook kinds increased; |
||||
3. whether virtual property declaration conditions and offsets changed; |
||||
4. whether `zend_hooked_object_get_iterator()` is still an exported API; |
||||
5. whether class linking, inheritance, variance, and Reflection added new required metadata; |
||||
6. whether the destruction and inheritance-copy rules for persistent internal functions changed. |
||||
|
||||
If Zend later provides an official extension registration API, migration to that API should be prioritized to reduce direct dependence on internal structure layout. |
||||
|
||||
## 9. Test requirements |
||||
|
||||
Property Hook changes must at least cover: |
||||
|
||||
- direct getter/setter and backing access; |
||||
- Reflection differences between virtual properties and backed properties; |
||||
- `hasHooks()`, `getHooks()`, Hook names, and final status; |
||||
- `get_object_vars()`, JSON, and object `foreach`; |
||||
- serialization containing only real stored state; |
||||
- dynamic Zend property reads/writes; |
||||
- inheritance and property visibility; |
||||
- PHP 8.4 and PHP 8.5 builds. |
||||
|
||||
Current core regression tests are located at: |
||||
|
||||
- `tests/compiler/object_property/property-hooks.phpt`; |
||||
- `tests/compiler/object_property/property-hooks-operations.phpt`; |
||||
- `tests/compiler/object_property/property-hooks-reflection.phpt`; |
||||
- `tests/compiler/object_property/property-hooks-introspection.phpt`. |
||||
@ -0,0 +1,151 @@ |
||||
# py2php: Python → TypePHP source conversion tool |
||||
|
||||
## Usage |
||||
|
||||
```bash |
||||
./bin/tpc.php --convert-python-to-php examples/python/version.py > examples/python/version.php |
||||
``` |
||||
|
||||
The generated PHP source is written to stdout, errors to stderr, with exit code 0 for success / 1 for failure. |
||||
|
||||
## Architecture |
||||
|
||||
``` |
||||
.py source |
||||
└─ PythonAstLoader python3 subprocess (ast module) → JSON AST |
||||
└─ PythonToTypePhpConverter AST → TypePHP source string |
||||
└─ Command::execute CLI dispatch (--convert-python-to-php) |
||||
``` |
||||
|
||||
- Source: `src/PythonTools/Command.php`, `src/PythonTools/Converter/` |
||||
- Unsupported syntax throws `RuntimeException("{file}:{line}: unsupported Python syntax {node type}[: details]")`, which the CLI layer converts to stderr + exit code 1. |
||||
- Tests: `phpunit/src/PythonTools/` (`PythonToTypePhpConverterTest`, `PythonAstLoaderTest`, `PythonToolsCommandTest`), corresponding item by item to this document. |
||||
|
||||
## Statement support matrix |
||||
|
||||
| Python syntax | Status | Conversion rule / error | |
||||
|---|---|---| |
||||
| `x = expr` | ✅ | `$x = expr;`, module-level variables are automatically injected as `global` | |
||||
| `x = y = 1` (chained assignment) | ✅ | `$x = $y = 1;` (name targets only; errors on property/subscript targets) | |
||||
| `x += expr` and other augmented assignments | ✅ | supports the `+ - * / % ** << >> \| ^ &` families; `//=` `@=` expand to `python\operator\floordiv/matmul($x, ...)` calls | |
||||
| `x: int = expr` | ✅ | annotation ignored, converted to an ordinary assignment | |
||||
| `x: int` (annotation only) | ✅ | converted to comment `// annotation-only declaration: x`, not registered as a module global | |
||||
| `a, b = x` (destructuring) | ✅ | `[$a, $b] = $x->toArray();` (PyObject converted to PHP array then destructured; elements may be names/properties/subscripts. Nested destructuring, star destructuring `a, *b = x`, and chained destructuring are not supported. Element-count mismatches fill `null` per PHP semantics rather than raising Python's ValueError) | |
||||
| `def f(...)` | ✅ | see "Function signatures"; a function named `main` is renamed to `main_` (to avoid conflict with the TypePHP entry point), and call sites are rewritten accordingly | |
||||
| nested `def` | ❌ | `FunctionDef: nested functions require Python closure scope analysis` | |
||||
| `@decorator` | ✅ | see "Function decorators" | |
||||
| `return [expr]` | ✅ | `return [expr];` | |
||||
| `if / elif / else` | ✅ | isomorphic conversion | |
||||
| `while` | ✅ | isomorphic conversion; `while/else` is not supported | |
||||
| `for i in iter` | ✅ | `foreach (iter as $i)`; `for/else` and tuple targets are not supported | |
||||
| `break` / `continue` / `pass` | ✅ | `pass` → `// pass` comment | |
||||
| `global x` | ✅ | `global $x;` (when combined with the auto-injected global it appears twice — redundant but valid, a known behavior) | |
||||
| `del x` / `del o.a` / `del d[k]` | ✅ | `unset(...)`; `del (a, b)` tuple/list targets expanded item by item; invalid del targets (such as `del f()`) are rejected first by the Python parser | |
||||
| module-level string literal (docstring) | ✅ | converted to `/** ... */` comment (`*/` escaped as `* /`) | |
||||
| `import a.b` | ✅ | `use python\a;` (only the first segment as the alias, see "Known behaviors") | |
||||
| `import a.b as x` | ✅ | `use python\a\b as x;` (`as` omitted when the alias equals the last segment) | |
||||
| `from m import f [as g]` | ✅ | call sites mapped to `python\m\f(...)` | |
||||
| `from . import m` | ❌ | `ImportFrom: relative imports are not supported yet` | |
||||
| `from m import *` | ❌ | `ImportFrom: star imports are not supported` | |
||||
| `class` | ❌ | `ClassDef` | |
||||
| `with` | ❌ | `With` | |
||||
| `raise` / `try` / `assert` | ❌ | `Raise` / `Try` / `Assert` | |
||||
| `async def` / `await` | ❌ | `AsyncFunctionDef` (`await` unreachable, outer level errors first) | |
||||
| `match` | ❌ | `Match` | |
||||
| `nonlocal` | ❌ | `Nonlocal` | |
||||
|
||||
## Function signatures |
||||
|
||||
| Python form | Status | TypePHP output | |
||||
|---|---|---| |
||||
| `def f(x, y=4)` | ✅ | `function f($x, $y = 4)` | |
||||
| `def f(a, *, b)` | ✅ | `function f($a, $b = null)` (keyword-only parameters without defaults are padded with `null`) | |
||||
| `def f(*args)` / `def f(**kw)` | ✅ | `function f(...$args)` | |
||||
| `def f(*a, **kw)` | ❌ | `FunctionDef: simultaneous *args and **kwargs cannot be represented by one PHP signature` | |
||||
| `lambda a, b=2: a + b` | ✅ | `fn ($a, $b = 2) => $a + $b` | |
||||
|
||||
## Expression support matrix |
||||
|
||||
| Python syntax | Status | Conversion rule / error | |
||||
|---|---|---| |
||||
| literals `int / float / str / True / False / None` | ✅ | `var_export`; `None` → `null` | |
||||
| `b'...'` bytes | ❌ | `{file}: Python bytes literals are not supported yet` (no line number) | |
||||
| `1j` complex | ❌ | `{file}: Python complex literals are not supported yet` (no line number) | |
||||
| variable names | ✅ | `$name`; `this` escaped as `$this_` | |
||||
| module alias as a value | ❌ | `a Python module cannot be used as a first-class value in TypePHP namespace syntax` | |
||||
| attribute chain `o.a.b` | ✅ | `$o->a->b`; for module alias chains only the first segment is a module member: `sys.version_info.major` → `sys\version_info->major` | |
||||
| module attribute assignment/deletion | ❌ | `Attribute: Python module attributes cannot be assigned or deleted` | |
||||
| function call | ✅ | defined functions connect directly `f(...)`; built-ins mapped `python\len(...)`; `from m import f` mapped `python\m\f(...)`; other names callable as variables `$f(...)` | |
||||
| keyword arguments / `*args` / `**kwargs` calls | ✅ | `f(x: 1, ...$args)` | |
||||
| container literals `[] () {} {:}` | ✅ | `python\list/tuple/set/dict([...])`, supports `...` unpacking | |
||||
| binary operators `+ - * / % ** << >> \| ^ &` | ✅ | isomorphic conversion | |
||||
| `//` floor division / `@` matrix multiplication | ✅ | `python\operator\floordiv(a, b)` / `python\operator\matmul(a, b)` | |
||||
| unary operators `- + not ~` | ✅ | `- + ! ~` | |
||||
| comparisons `== != < <= > >=` | ✅ | isomorphic conversion | |
||||
| `is` / `is not` | ✅ | `===` / `!==` | |
||||
| `in` / `not in` | ✅ | `python\operator\contains(b, a)` (arguments swapped) / negated | |
||||
| chained comparison `a < b < c` | ❌ | `Compare: chained comparisons require explicit temporary variables` | |
||||
| `a and b` / `a or b` | ❌ | `BoolOp` | |
||||
| `x if c else y` | ✅ | `(c ? x : y)` | |
||||
| subscript `a[i]` / slice `a[l:u:s]` | ✅ | `$a[$i]` / `$a[python\slice(l, u, s)]` (defaults to `null`) | |
||||
| f-string | ✅ | concatenation + `->toString()`; operator-precedence-sensitive expressions are parenthesized as a whole | |
||||
| f-string `!r` conversion / `:03d` format spec | ❌ | `FormattedValue: formatted f-string conversions are not supported yet` | |
||||
| walrus `:=` | ✅ | assignment within expression `($n = 10)` | |
||||
| comprehensions / generator expressions | ❌ | `ListComp` / `SetComp` / `DictComp` / `GeneratorExp` | |
||||
| `yield` / `yield from` | ❌ | `Yield` / `YieldFrom` | |
||||
|
||||
## Function decorators |
||||
|
||||
Decorators rebind the function to the same-named module variable at the start of `main()` (before other top-level statements), bottom-up per Python semantics: |
||||
|
||||
```python |
||||
@a |
||||
@b |
||||
def greet(): ... |
||||
``` |
||||
|
||||
```php |
||||
function greet() { ... } |
||||
|
||||
function main(): void |
||||
{ |
||||
global $greet; |
||||
$greet = b('greet'); |
||||
$greet = a('greet'); |
||||
... |
||||
} |
||||
``` |
||||
|
||||
- A decorator can be a defined function, a `from m import f` imported symbol, a module attribute, or a decorator factory (`@dec('x')` → `$greet = dec('x')('greet');`) |
||||
- The decorated function name is registered as a module global, and all call sites (including inside other function bodies) call the decorated result indirectly via `global` + variable: `$greet()` |
||||
- Recursive calls inside the decorated function body also resolve to the decorated variable, consistent with Python semantics |
||||
|
||||
## print / sys.exit degradation rules |
||||
|
||||
Degrade to native statements only when PHP behavior is fully identical to Python: |
||||
|
||||
| Form | Output | |
||||
|---|---| |
||||
| `print()` | `echo "\n";` | |
||||
| `print("a", "b")` (string/integer constants, module attributes, containers, f-strings) | `echo 'a', ' ', 'b', "\n";` | |
||||
| `print(1.5)`, `print(True)`, `print(x, sep=...)` | not degraded: `python\print(...)` | |
||||
| after user-defined/imported/assigned shadowing of `print` | not degraded | |
||||
| `sys.exit()` / `sys.exit(2)` (including the `from sys import exit` form) | `exit;` / `exit(2);` | |
||||
| `sys.exit("fail")` | not degraded: `sys\exit('fail');` | |
||||
|
||||
## Known behaviors (not errors, but worth noting) |
||||
|
||||
1. `import os.path` (no alias) only introduces the first segment `use python\os;`. |
||||
2. An explicit `global x` inside a function and the auto-injected `global x` for a module global appear twice (valid PHP). |
||||
3. Writing `print = str` (assigning a built-in name to a variable) treats the right side as a variable (`$print = $str;`), not as built-in name resolution. |
||||
4. Errors for bytes/complex literals have no line number (constants are encoded during the AST load stage; position information is not passed through). |
||||
5. Decorator rebinding uniformly happens at the start of `main()`, slightly differing from Python's exact "decorated at the def site" position; if a decorator expression depends on assignments later in the top-level statements, the evaluation timing may differ. |
||||
6. Decorated function names are registered as module globals, so the name appears in every function's auto-injected `global` list (redundant but valid). |
||||
|
||||
## Running tests |
||||
|
||||
```bash |
||||
vendor/bin/phpunit --filter 'PythonToTypePhpConverterTest|PythonAstLoaderTest|PythonToolsCommandTest' |
||||
``` |
||||
|
||||
Converter tests depend on a real `python3` to parse the AST, and are skipped automatically when the environment lacks it. |
||||
@ -0,0 +1,48 @@ |
||||
# TypePHP Compiler Internal Documentation |
||||
|
||||
This directory contains compiler implementation, compatibility, build-mode, and special-topic design documents. The user-facing manual lives in the separate `aot/docs` repository; the research reports and refactoring plans here may describe historical state, and current behavior should be determined by the code, tests, and compatibility checklist. |
||||
|
||||
## Current authoritative documents |
||||
|
||||
- [AOT and PHP Incompatible Features Checklist](INCOMPATIBLE_PHP_FEATURES.md): a concise list of current limitations. |
||||
- [Incompatibility Classification](PHP_INCOMPATIBILITY_CLASSIFICATION.md): distinguishes Hard Limit, Intentional Rule, Pending, and Partial. |
||||
- [Compiler CLI](COMPILER_CLI.md): current CLI arguments and project configuration. |
||||
- [Compilation Modes](COMPILATION_MODES.md): binary, extension, library modes. |
||||
- [Quick Start](QUICKSTART.md): the minimal compile flow. |
||||
- [Compile-time Functions](COMPILE_TIME_FUNCTIONS.md): `any()`, `refval()`, `objval()`, `expected()`, `unexpected()`, and keyword methods. |
||||
- [Native Types](NATIVE_TYPES.md), [High-Precision Types](HIGH_PRECISION_TYPES.md), [Std Containers](STD_CONTAINERS.md). |
||||
- [Three Object Storage and Passing Models](OBJECT_STORAGE_AND_PASSING_MODELS.md): the responsibilities, ABI, and non-substitutable boundaries of Zend Object, PHPX Box, and Native Class Object. |
||||
- [Universal and Extension Methods](UNIVERSAL_METHODS.md), [Generator](YIELD_GENERATOR.md). |
||||
- [`#[Immutable]` compile-time read-only contract](IMMUTABLE.md): methods, parameters, aliases, call boundaries, and dynamic escape rules. |
||||
- [`#[ArrayDef]` array property contract](ARRAY_DEF.md): List/Map metadata, direct-write checks, and dynamic escape boundaries. |
||||
- [Class Inheritance](CLASS_INHERITANCE.md), [Mixed C++/PHP](MIXED_CPP_PHP.md). |
||||
|
||||
## Architecture and maintenance |
||||
|
||||
- [Backend-Neutral IR](BACKEND_NEUTRAL_IR.md) |
||||
- [TypePHP WASM Technical Plan and Implementation Plan](TYPEPHP_WASM_IMPLEMENTATION_PLAN.md) |
||||
- [Building TypePHP WASI Programs](WASI_BUILD.md) |
||||
- [Rebuilding the PHPX WASM Static Library](PHPX_WASM_BUILD.md): incremental rebuild of `libphpx.a`, numeric-dependency rebuild, and full SDK rebuild boundaries. |
||||
- [Core Refactoring Plan](REFACTORING_PLAN.md) |
||||
- [Scope Management Design](SCOPE_MANAGEMENT.md): responsibilities and usage boundaries of `CallableScope`, `UserCodeScopeGuard`, and `FakeScopeGuard`. |
||||
- [Runtime Initialization and Shutdown Flow](RUNTIME_LIFECYCLE.html): the four-layer lifecycle of PHP, PHPX, TypePHP, and the project, covering bin/ext/lib, multi-module, and WASM. |
||||
- [C++ Namespaces, Prefixes, and Symbol ABI](CPP_SYMBOL_NAMING.md): responsibility boundaries and conflict rules for `typephp_`, `php::`, `typephp_<project>`, and user callables `php_`. |
||||
- [Zend Object Creation and Property Default Value Initialization](OBJECT_CREATION.md): the `gen_stub.php` default property table, trigger conditions for custom `create_object`, execution flow, and performance boundaries. |
||||
- [Native Class Object Design](NATIVE_CLASS_OBJECT.md) and [Implementation Acceptance Matrix](NATIVE_CLASS_IMPLEMENTATION_AUDIT.md). |
||||
- [PHP 8.4 Property Hook Integration Design](PROPERTY_HOOKS.md): compile-time lowering, Zend Hook metadata, object introspection, and PHPX ABI boundaries. |
||||
- [Interface Property Hook Implementation Plan](INTERFACE_PROPERTY_HOOKS.md): interface property contracts, compile-time variance checks, and PHP 8.4 abstract Hook metadata. |
||||
- [Build Speed Research](AOT_BUILD_SPEED_RESEARCH.md) |
||||
- [Optimization Priority](aot-optimization-priority.md) |
||||
- [High-Precision Type In-Place Operation Optimization Plan](BIG_NUMBER_INPLACE_OPTIMIZATION_PLAN.md) |
||||
- [GMP Differences](GMP_GAP.md) |
||||
|
||||
## Research and historical materials |
||||
|
||||
`hhvm-review.md`, `kphp-review.md`, `peachpie-review.md`, `phpstan-design-analysis.md`, `php-src-optimizer-analysis.md`, and the patent drafts record comparisons and design background at the time of investigation, and do not serve as the current feature list. |
||||
|
||||
## Maintenance rules |
||||
|
||||
1. When current compatibility changes, update `INCOMPATIBLE_PHP_FEATURES.md` and the classification document simultaneously. |
||||
2. All syntax and semantic limitations link uniformly to the current compatibility checklist to avoid maintaining duplicate lists. |
||||
3. Feature support should be determined by PHPT/PHPUnit regression tests. |
||||
4. Historical research documents preserve the original comparison conclusions and note the investigation date where necessary; they should not be silently rewritten to reflect the current state. |
||||
@ -0,0 +1,430 @@ |
||||
# AOT Compiler Core Refactoring Plan |
||||
|
||||
> For the next-phase OOA/OOD/OOP refactoring of `Translator`, `CompilerBase`, and `Preprocessor`, please use [CORE_OOA_OOD_OOP_REFACTORING_PLAN.md](CORE_OOA_OOD_OOP_REFACTORING_PLAN.md) as the implementation baseline. This document retains the historical plan of the earlier modularization refactoring. |
||||
|
||||
## Background |
||||
|
||||
The current core classes of the AOT compiler carry too many responsibilities. In particular, classes such as `CompilerBase` and `Translator` simultaneously contain AST dispatch, type inference, property access resolution, call resolution, code generation, diagnostics, and context state maintenance. As functionality continues to grow, this structure causes the following problems: |
||||
|
||||
- Insufficient encapsulation: modifying one semantic point easily affects multiple code paths. |
||||
- Insufficient code reuse: similar logic is repeatedly implemented across normal properties, static properties, nullsafe, assignment, isset/empty/refval, and other paths. |
||||
- Compile-time checks are prone to bypass paths: for example, some dynamic fallbacks do not reuse the static resolver. |
||||
- Individual classes are too large, and review, test localization, and long-term maintenance costs keep rising. |
||||
- Design boundaries are unclear: the type system, symbol resolution, property access, and call generation are too deeply coupled. |
||||
|
||||
This plan guides the subsequent incremental refactoring, aiming to improve architectural quality while avoiding the behavior regression risk of a large-scale one-shot rewrite. |
||||
|
||||
## Core Principles |
||||
|
||||
1. Incremental refactoring; a one-shot rewrite of the core compilation flow is forbidden. |
||||
2. Prioritize extracting pure logic with no or low state dependency, then logic that depends on the compilation context. |
||||
3. Each refactoring step should keep behavior unchanged as much as possible; behavior changes must be explained separately and covered by tests. |
||||
4. New modules should be modeled around a stable domain, not mechanically split by AST node. |
||||
5. Prefer small services/helpers, explicit DTOs, resolvers, and emitters; introduce patterns such as Visitor and Strategy only after boundaries stabilize. |
||||
6. All error messages remain PHP-style, do not expose implementation details, and do not use expressions like "AOT forbids". |
||||
7. Each phase must have phpunit or phpt regression verification, especially for high-risk paths such as properties, types, calls, inheritance, and exceptions. |
||||
8. Compiler interfaces exposed to resolver/emitter should preferably remain read-only. Read-only query interfaces can be made public as needed, but write operations must be handled with extra care to avoid bypassing unified state management. |
||||
|
||||
## Target Architecture Direction |
||||
|
||||
### CompilerBase |
||||
|
||||
It should ultimately converge into the compilation context, common utilities, AST dispatch entry point, and cross-module collaboration layer, no longer directly carrying a large amount of domain logic. |
||||
|
||||
Responsibilities to retain: |
||||
|
||||
- Current file, function, class, and method context management. |
||||
- Temporary variable, local variable, and scope state maintenance. |
||||
- Top-level AST dispatch entry point. |
||||
- Unified entry point for fatal/warning. |
||||
- Collaboration with lower-level resolver/emitter. |
||||
|
||||
Responsibilities to gradually move out: |
||||
|
||||
- Type declaration parsing and type compatibility determination. |
||||
- Property visibility, property offset, and typed property checks. |
||||
- Function and method call resolution. |
||||
- Complex expression code generation. |
||||
- Union/intersection/nullable runtime typecheck generation. |
||||
|
||||
### Translator |
||||
|
||||
Retain file-level, class-level, and function-level compilation flow control, and gradually reduce specific semantic checking and expression generation logic. |
||||
|
||||
Responsibilities to retain: |
||||
|
||||
- File scanning and translation entry point. |
||||
- Class, function, and method code block generation flow. |
||||
- class/function/constant/property metadata registration. |
||||
- Compilation artifact organization. |
||||
|
||||
Responsibilities to gradually move out: |
||||
|
||||
- Detail logic of inheritance compatibility checks. |
||||
- Trait property conflict check details. |
||||
- Parameter, return value, and property type compatibility determination. |
||||
|
||||
## Module Splitting Plan |
||||
|
||||
### 1. TypeSystem |
||||
|
||||
Responsibilities: |
||||
|
||||
- Type declaration parsing. |
||||
- Mapping from PHP types to AOT internal types. |
||||
- Nullable, union, and intersection type expansion. |
||||
- Type compatibility determination for parameters, return values, properties, and constants. |
||||
- Boundary definition between static types and runtime typecheck. |
||||
- Type string formatting. |
||||
|
||||
Recommended submodules: |
||||
|
||||
- `TypeResolver` |
||||
- `TypeCompatibility` |
||||
- `TypeCheckEmitter` |
||||
- `TypeStringFormatter` |
||||
|
||||
Design requirements: |
||||
|
||||
- union, intersection, and nullable can still be treated as mixed/any in the static phase, but runtime typecheck information must be preserved. |
||||
- The resolution rules for special type names such as `self`, `parent`, and `static` must be centralized to avoid inconsistent implementation across multiple paths. |
||||
- Type error messages must include necessary context such as function, method, parameter, and property. |
||||
|
||||
### 2. SymbolResolver |
||||
|
||||
Responsibilities: |
||||
|
||||
- Namespace resolution. |
||||
- use alias resolution. |
||||
- `self`, `parent`, `static` resolution. |
||||
- Class, interface, trait, function, and constant name normalization. |
||||
- Boundary determination between dynamic symbols and statically resolvable symbols. |
||||
|
||||
Recommended interfaces: |
||||
|
||||
- `resolveClassName(NodeAbstract $node): SymbolResolution` |
||||
- `resolveFunctionName(NodeAbstract $node): SymbolResolution` |
||||
- `resolveMethodScope(NodeAbstract $node): SymbolResolution` |
||||
- `resolveClassConstScope(NodeAbstract $node): SymbolResolution` |
||||
|
||||
Design requirements: |
||||
|
||||
- Different call paths must not reimplement the `self/parent/static` rules. |
||||
- For symbols that cannot be statically determined, explicitly return a dynamic state instead of silently degrading to string concatenation. |
||||
|
||||
### 3. PropertyAccessResolver |
||||
|
||||
Responsibilities: |
||||
|
||||
- Object property access resolution. |
||||
- Static property access resolution. |
||||
- Nullsafe property access static checking. |
||||
- private/protected/public visibility checking. |
||||
- Native property offset lookup. |
||||
- Entry point for typed property write typecheck information generation. |
||||
- Static-vs-instance property misuse checking. |
||||
|
||||
Recommended interfaces: |
||||
|
||||
- `resolveInstancePropertyAccess(PropertyAccessRequest $request): PropertyAccessResult` |
||||
- `resolveStaticPropertyAccess(StaticPropertyAccessRequest $request): PropertyAccessResult` |
||||
- `assertReadable(PropertyAccessResult $result): void` |
||||
- `assertWritable(PropertyAccessResult $result): void` |
||||
- `emitRead(PropertyAccessResult $result): string` |
||||
- `emitWrite(PropertyAccessResult $result, string $value): string` |
||||
|
||||
Paths that need unified coverage: |
||||
|
||||
- `$obj->prop` |
||||
- `$obj?->prop` |
||||
- `Class::$prop` |
||||
- `self::$prop` |
||||
- `parent::$prop` |
||||
- `static::$prop` |
||||
- `isset($obj->prop)` |
||||
- `empty($obj->prop)` |
||||
- `refval($obj->prop)` |
||||
- normal assignment, compound assignment, increment/decrement, unset. |
||||
|
||||
It is recommended to start from this module as the first priority, because recent problems are concentrated in property access and visibility bypass, and the test boundaries are relatively clear. |
||||
|
||||
### 4. CallResolver |
||||
|
||||
Responsibilities: |
||||
|
||||
- Function call resolution. |
||||
- Object method call resolution. |
||||
- Static method call resolution. |
||||
- Native call vs. dynamic call selection. |
||||
- Named args, unpack, and by-ref parameter handling. |
||||
- Closure and dynamic callable degradation rules. |
||||
|
||||
Recommended interfaces: |
||||
|
||||
- `resolveFunctionCall(CallRequest $request): CallResolution` |
||||
- `resolveMethodCall(MethodCallRequest $request): CallResolution` |
||||
- `resolveStaticCall(StaticCallRequest $request): CallResolution` |
||||
- `emitCall(CallResolution $resolution): string` |
||||
|
||||
Design requirements: |
||||
|
||||
- When the parameter information of a static function or built-in function is clear, references can be automatically converted. |
||||
- For dynamic calls, closures, and cases where by-ref parameter information cannot be obtained at compile time, an explicit `refval()` must be required. |
||||
- When using unpack with trailing named args appended, it should degrade to a dynamic call and must not go through a native call. |
||||
|
||||
### 5. ExpressionEmitter |
||||
|
||||
Responsibilities: |
||||
|
||||
- Expression-level code generation. |
||||
- Gradually split the large `parseExpr()` dispatch logic. |
||||
- Reuse resolver results to generate C++ code. |
||||
|
||||
It is recommended to split by domain rather than starting with a large number of AST visitors: |
||||
|
||||
- `AssignmentEmitter` |
||||
- `PropertyEmitter` |
||||
- `CallEmitter` |
||||
- `ArrayEmitter` |
||||
- `ControlExprEmitter` |
||||
- `ObjectEmitter` |
||||
|
||||
Design requirements: |
||||
|
||||
- Keep the existing `parseExpr()` as the dispatch entry point for now. |
||||
- Migrate only one group of expressions at a time, and run the corresponding test group after each migration. |
||||
- Handle expression side effects, evaluation order, and temporary variable generation conservatively. |
||||
|
||||
### 6. Diagnostic |
||||
|
||||
Responsibilities: |
||||
|
||||
- Unified fatal/warning construction. |
||||
- Provide context enhancement capability. |
||||
- Ensure error message style is close to PHP. |
||||
|
||||
Recommended capabilities: |
||||
|
||||
- Current function/method name. |
||||
- Parameter name. |
||||
- Property name and class name. |
||||
- Source location. |
||||
- Declaration location and usage location. |
||||
|
||||
Design requirements: |
||||
|
||||
- Error messages must not use "AOT" as the actor. |
||||
- For user-fixable problems, accurate symbol names should be included. |
||||
- Problems discoverable at compile time should preferably be compile-time fatal, and should not rely on runtime typecheck exceptions as a fallback. |
||||
|
||||
## Phase Plan |
||||
|
||||
### Phase 1: Property Access Resolution Modularization |
||||
|
||||
Objectives: |
||||
|
||||
- Extract property access resolution logic such as `findNativeProperty()`, `findNativeStaticProperty()`, `canAccessProtectedProperty()`. |
||||
- Keep generated code essentially unchanged. |
||||
- Establish a unified `PropertyAccessResult` carrying information such as property declaration, declaring class, accessing class, whether native, offset, and whether dynamic. |
||||
|
||||
Scope: |
||||
|
||||
- Normal object property reads. |
||||
- Static property reads. |
||||
- Nullsafe property static checking. |
||||
- Property visibility checking. |
||||
|
||||
Current progress: |
||||
|
||||
- Established `PropertyAccessResolver` and `PropertyAccessResult` as the first abstraction layer for property access resolution. |
||||
- Instance property reads now complete native property lookup, static-vs-instance checking, and visibility checking through the resolver's explicit `resolveNativeInstanceProperty()` interface. |
||||
- `findNativeStaticProperty()` now completes static property checking through the resolver's explicit `resolveNativeStaticProperty()` interface. |
||||
- Nullsafe property chain checking now completes class-name advancement and visibility checking through the resolver's explicit `resolveNullsafePropertyChain()` interface. |
||||
- The old generic `CompilerBase::findNativeProperty()` entry point has been removed to prevent further spread of access patterns carrying the `$static` boolean parameter. |
||||
- The old `findNativeStaticProperty(..., &$class)` by-ref protocol has been removed; static property reads now use explicit DTOs `StaticPropertyFetchTarget` and `StaticPropertyFetchResolution`. |
||||
- Target class resolution for instance property reads has been extracted to `InstancePropertyFetchTarget`; `getPropertyIdentifier()` no longer mixes target resolution, resolver calls, and dynamic fallback branches. |
||||
- The `nativeProperty`, `nativePropertyDef`, and `nativeClassDef` previously scattered on AST attributes have been merged into the `NativePropertyAccess` metadata to avoid inconsistent state among the three. |
||||
- Direct reads and writes of `nativePropertyVar`, `nativePropertyValueSource`, `objectProps`, and `staticPropRefs` have been converged into helper methods; business paths no longer determine property access semantics by string content. |
||||
- Typed instance property hoist and typed static property ref registration have been extracted into independent helpers, currently still keeping the original generated code structure. |
||||
- `CompilerBase::isSameClassName()`, `isSameOrSubclassOf()`, and `canAccessProtectedProperty()` have been delegated to the resolver to avoid further rule spread. |
||||
- Added `prepare/convert/idle` compilation phase states; `PropertyAccessResolver` can only be created and used in the convert phase to avoid misusing incomplete class table state in the preprocessing phase. |
||||
- `PropertyAccessResolver` has been changed to depend on the read-only `PropertyAccessContext` interface instead of fully depending on the large `CompilerBase` class. |
||||
- Established `PropertyAssignTypeInfo`, extracting the pure metadata computation for typed property writes, including fixed-type property determination, default values, the runtime typecheck list, and type strings. |
||||
- The current migration keeps generated code unchanged; read/write emitters will be unified in subsequent phases. |
||||
|
||||
Status: |
||||
|
||||
- Phase 1 is essentially wrapped up. Unless property read resolver bypass or behavior regression is found later, the scope of Phase 1 will not be further expanded. |
||||
- Phase 2 has begun; assignment, compound assignment, inc/dec, unset, and refval paths related to property writes still need to be further unified. |
||||
|
||||
Verification: |
||||
|
||||
- `phpunit/src/NativePropertyTest.php` |
||||
- `phpunit/src/InheritanceErrorTest.php` |
||||
- object property related phpt. |
||||
- static property related phpt. |
||||
- nullsafe related phpt. |
||||
|
||||
### Phase 2: Property Write Path Unification |
||||
|
||||
Objectives: |
||||
|
||||
- All property write paths resolve first, then emit. |
||||
- Typed property runtime typecheck converges from scattered logic into the property write module. |
||||
- Eliminate the problem where assignment, compound assignment, inc/dec, and unset each implement property access rules independently. |
||||
|
||||
Scope: |
||||
|
||||
- `$obj->prop = $value` |
||||
- `$obj->prop += $value` |
||||
- `$obj->prop++` |
||||
- `unset($obj->prop)` |
||||
- `Class::$prop = $value` |
||||
- `??=` related property paths. |
||||
|
||||
Verification: |
||||
|
||||
- typed property related phpunit/phpt. |
||||
- object property optimization related phpt. |
||||
- nullsafe write context error tests. |
||||
- private/protected/static property error tests. |
||||
|
||||
Current progress: |
||||
|
||||
- Phase 2 has begun. |
||||
- Established `PropertyWriteTarget` as the minimal target DTO for the property write path. |
||||
- Normal assignment and `??=` have been connected to `preparePropertyWriteTarget()`, uniformly completing property target preparation before writes, and executing static checks and runtime typecheck wrapping through `assertCanAssignPropertyWrite()` and `wrapPropertyWriteTypeCheck()`. |
||||
- `getProperty()` / `setProperty()` generation for dynamic object properties has been converged into the `emitDynamicPropertyRead()` / `emitDynamicPropertyWrite()` helpers; normal dynamic property assignment, compound assignment, and increment/decrement now reuse this entry point. |
||||
- The dynamic property path of compound assignment has been connected to `preparePropertyWriteTarget()`, uniformly completing property write target preparation and static checks first. |
||||
- `PropertyWriteTarget` has begun carrying the object/property expressions of safe dynamic property write targets; normal dynamic property assignment, compound assignment, and increment/decrement now prefer emitting code through target-level read/write helpers. |
||||
- Dynamic property `unset`, property array dimension writes, and safe object property reference paths in reference arguments/refval/reference assignment have begun reusing target-level unset/ref helpers. |
||||
- Target/ref generation for object property reference expressions has been converged into `emitDynamicPropertyFetchRef()`; the unused old static property assignment entry point has been deleted, and static property assignment continues through the unified assignment target path. |
||||
- The dynamic object/property fields of `PropertyWriteTarget` have been encapsulated as getters; property array dimension writes have been connected to target-level append/update emitters. |
||||
- Established the `emitDynamicPropertyFetchRead/Write/Unset/AppendArray/UpdateArray()` wrapper layer; callers only pass in the property access AST and an optional target, and `CompilerBase` uniformly selects the target path or the old fallback path. |
||||
- Normal assignment, compound assignment, increment/decrement, unset, property array dimension writes, and reference assignment have removed the direct branch determination of dynamic targets in the Parser trait, instead reusing the unified emitter wrappers. |
||||
- To avoid changing the evaluation order of complex expressions, currently only dynamic property writes whose object part is a variable have their target object/property fields populated; complex object expressions still retain the old path. |
||||
- The current step keeps generation logic compatible for valid code, but will route more property write paths into unified static checking; continue converging static/native property write emitters and `??=` property write result generation. |
||||
|
||||
### Phase 3: Type System Modularization |
||||
|
||||
Objectives: |
||||
|
||||
- Extract type declaration parsing, type compatibility, and runtime typecheck generation. |
||||
- Clarify the responsibility boundary between static type inference and runtime typecheck. |
||||
- Reduce the repeated handling of type rules across parameters, return values, properties, and constants. |
||||
|
||||
Scope: |
||||
|
||||
- `parseTypeDecl()`. |
||||
- `buildTypeCheckFromNode()`. |
||||
- Parameter typecheck. |
||||
- Return value typecheck. |
||||
- Property typecheck. |
||||
- Constant type handling. |
||||
|
||||
Verification: |
||||
|
||||
- union, intersection, nullable type tests. |
||||
- Parameter, return value, property typecheck tests. |
||||
- namespace constant tests. |
||||
- constructor, void expression related tests. |
||||
|
||||
### Phase 4: Call Resolution Modularization |
||||
|
||||
Objectives: |
||||
|
||||
- Unify function, method, and static method call resolution. |
||||
- Clarify degradation rules for native calls, dynamic calls, and closure calls. |
||||
- Centrally handle named args, unpack, and by-ref parameters. |
||||
|
||||
Scope: |
||||
|
||||
- `parseFuncCall()`. |
||||
- `parseMethodCall()`. |
||||
- `parseStaticCall()`. |
||||
- call args parsing. |
||||
- native/internal/user function call paths. |
||||
|
||||
Verification: |
||||
|
||||
- named args, unpack related phpt. |
||||
- by-ref parameter related phpt. |
||||
- closure related phpt. |
||||
- parent/self/static call related phpt. |
||||
|
||||
### Phase 5: Expression Generator Splitting |
||||
|
||||
Objectives: |
||||
|
||||
- Migrate the large amount of expression generation logic behind `parseExpr()` to domain emitters. |
||||
- `CompilerBase` retains dispatch and shared context capabilities. |
||||
- Reduce the code size of individual files and classes. |
||||
|
||||
Scope: |
||||
|
||||
- AssignmentEmitter. |
||||
- PropertyEmitter. |
||||
- CallEmitter. |
||||
- ArrayEmitter. |
||||
- ControlExprEmitter. |
||||
|
||||
Verification: |
||||
|
||||
- Run the corresponding test group after each emitter migration. |
||||
- Finally run the core phpunit and selected phpt regression set. |
||||
|
||||
### Phase 6: Translator Convergence |
||||
|
||||
Objectives: |
||||
|
||||
- Extract inheritance compatibility, trait merging, and class member validation logic into dedicated checkers. |
||||
- `Translator` focuses on compilation flow organization. |
||||
|
||||
Recommended submodules: |
||||
|
||||
- `InheritanceChecker` |
||||
- `TraitCompositionChecker` |
||||
- `ClassMemberValidator` |
||||
- `FunctionSignatureChecker` |
||||
|
||||
Verification: |
||||
|
||||
- inheritance error phpunit. |
||||
- trait related phpt. |
||||
- interface/abstract/final/readonly related phpt. |
||||
|
||||
## Test Gate |
||||
|
||||
Each refactoring PR or phase must at least satisfy: |
||||
|
||||
- Relevant phpunit must pass. |
||||
- Relevant phpt must pass. |
||||
- If C++/phpx is modified, gtest must be supplemented and the corresponding tests must pass. |
||||
- If a new compile-time error is introduced, a fixed fixture must be added to `phpunit/code` or a new phpt must be added. |
||||
- Do not use `file_put_contents()` to temporarily generate source code as a new test method. |
||||
|
||||
It is recommended to maintain a minimal regression set per module: |
||||
|
||||
- Property module: `NativePropertyTest`, `InheritanceErrorTest`, object property, static property, nullsafe. |
||||
- Type module: type_decl, type_hits, typed property, union/intersection/nullable. |
||||
- Call module: function call, method call, parent_call, closure, named args, unpack, by-ref. |
||||
- Control flow and expressions: ternary, match, goto, loop, array, coalesce, void expression. |
||||
|
||||
## Risk Control |
||||
|
||||
- Do not perform large-scale file movement and behavior changes in the same change. |
||||
- Supplement tests for current behavior before each migration, especially the paths corresponding to historical bugs. |
||||
- Keep old entry points for a period, calling new modules through adapters to reduce switching risk. |
||||
- Remain conservative about dynamic PHP semantics; do not over-optimize when it cannot be statically determined. |
||||
- For places where AOT is clearly incompatible with PHP's historical baggage, express them as language rules in documentation and error messages, not as implementation limitations. |
||||
|
||||
## Recommended Next Step |
||||
|
||||
Start from `PropertyAccessResolver`. |
||||
|
||||
Reasons: |
||||
|
||||
- Recent bugs are mostly concentrated in property access, visibility, typed property, nullsafe, and static-vs-instance paths. |
||||
- Existing tests are relatively easy to extend. |
||||
- Property access is a relatively independent domain outside the type system, optimizer, and call generation, making it suitable for establishing the resolver/result pattern first. |
||||
- After completion, it can directly reduce the complex branches in `CompilerBase` and lay the foundation for subsequent ExpressionEmitter splitting. |
||||
@ -0,0 +1,401 @@ |
||||
# TypePHP Scope Management Design |
||||
|
||||
This document is an internal implementation document for TypePHP and PHPX. It explains the responsibilities, implementation, lifecycle, performance characteristics, and applicable scenarios of the three current scope managers. The "scope" here is not a single Zend concept: callable resolution, execution-frame class scope, and `EG(fake_scope)` each serve different subsystems and cannot be substituted for one another. |
||||
|
||||
## 1. Design Goals |
||||
|
||||
The C++ methods generated by TypePHP are not ordinary Zend user functions. When a dynamic call returns to the ZendVM, Zend still needs the following information to reproduce PHP's visibility rules: |
||||
|
||||
- The lexical scope of the declared method, used to determine whether private/protected members are accessible; |
||||
- The called scope of the current late static binding; |
||||
- The current instance `$this`, used to resolve non-static method callables; |
||||
- The `EG(fake_scope)` read by certain Zend property, object, and exception APIs. |
||||
|
||||
The Scope design follows these principles: |
||||
|
||||
1. Prefer passing scope explicitly, and do not modify Zend's global or real execution-frame state. |
||||
2. Create a reusable callable context at most once per AOT method call; multiple calls within a loop share it. |
||||
3. Temporarily modify the nearest user-code frame only when the compiler cannot determine the callback location. |
||||
4. Use RAII when modifying Zend executor state, and guarantee restoration on the exception path. |
||||
5. Do not pay extra wrapping cost for pure Native Calls or public, absolutely-located callbacks. |
||||
|
||||
## 2. Overview |
||||
|
||||
| Manager | Managed State | Primary Purpose | Modifies Zend Current State | |
||||
| --- | --- | --- | --- | |
||||
| `php::CallableScope` | A synthetic `zend_execute_data` containing lexical scope, called scope, and `$this` | Dynamic method calls, first-class callables, built-in function callbacks | No | |
||||
| `php::UserCodeScopeGuard` | The `zend_function::common.scope` of the nearest user-code frame | `call_user_func*` and dynamic call paths where the callback is hidden inside argument unpacking | Yes, restored on destruction | |
||||
| `php::FakeScopeGuard` | `EG(fake_scope)` | Zend property, object, exception, and other APIs that read fake scope | Yes, restored on destruction or explicit `restore()` | |
||||
|
||||
The selection rule can be simplified as: |
||||
|
||||
- A concrete callable value is available: use `CallableScope`. |
||||
- Calling `call_user_func*`, or the callback of another built-in function is hidden in `...$args`: use `UserCodeScopeGuard`. |
||||
- The called Zend API explicitly reads `EG(fake_scope)`: use `FakeScopeGuard`. |
||||
- Pure native calls or operations that do not depend on caller visibility: do not create any Scope manager. |
||||
|
||||
## 3. `php::CallableScope` |
||||
|
||||
### 3.1 Responsibilities |
||||
|
||||
`CallableScope` is the main path for ordinary callable resolution today. It explicitly hands the caller context to `zend_is_callable_at_frame()` to: |
||||
|
||||
- Resolve private/protected methods; |
||||
- Resolve `self`, `parent`, and `static` callbacks; |
||||
- Preserve the called scope of late static binding; |
||||
- Provide the real `$this` for non-static methods; |
||||
- Invoke dynamic methods without modifying `EG(current_execute_data)` or the real execution frame. |
||||
|
||||
It does not handle property access and does not set `EG(fake_scope)`. |
||||
|
||||
### 3.2 Internal Structure |
||||
|
||||
The class is defined in PHPX's `include/phpx.h` and holds: |
||||
|
||||
```cpp |
||||
zend_function *caller_function_; |
||||
zend_class_entry *called_scope_; |
||||
zend_object *this_object_; |
||||
mutable zend_execute_data frame_{}; |
||||
``` |
||||
|
||||
On construction, it initializes a synthetic frame through `zend_vm_init_call_frame()`: |
||||
|
||||
- `caller_function_->common.scope` is the lexical scope, i.e. the class that declares the current method; |
||||
- `called_scope_` is the runtime called scope; |
||||
- Instance calls set `ZEND_CALL_HAS_THIS` and carry the real `zend_object *`; |
||||
- Static calls do not carry an object and only pass the called scope; |
||||
- If the called scope is empty, it falls back to the lexical scope. |
||||
|
||||
Resolution calls: |
||||
|
||||
```cpp |
||||
zend_is_callable_at_frame(callable, object, &frame_, 0, cache, error); |
||||
``` |
||||
|
||||
The synthetic frame is not installed into `EG(current_execute_data)`, so it does not pollute the current Zend call stack, and global state does not need to be restored on exit. |
||||
|
||||
### 3.3 Lifecycle and Ownership |
||||
|
||||
`CallableScope` does not own `zend_function`, `zend_class_entry`, or `zend_object`; it only borrows these pointers within the current AOT method stack frame: |
||||
|
||||
- TypePHP-compiled methods use persistent `zend_function`, whose lifecycle spans the request invocation; |
||||
- A Closure's `zend_function *` is valid for the lifetime of the Closure object; |
||||
- `$this` is valid during execution of the current method; |
||||
- `CallableScope` is non-copyable and non-movable, preventing the synthetic frame from being accidentally transferred or stored across lifecycles. |
||||
|
||||
`CallableScope` must not be cached beyond the request, nor be allowed to outlive its owning method or Closure. |
||||
|
||||
### 3.4 Compiler Generation Pattern |
||||
|
||||
The compiler lazily requests a Scope variable through `FunctionContext::$callableScopeVar`. The first time an explicit callable scope is needed, `getCallableScopeExpr()` allocates a temporary variable; subsequently `genScopeVarDecl()` hoists the initialization code to the function entry: |
||||
|
||||
```cpp |
||||
php::CallableScope tmp_var_1 = php::getCallableScope( |
||||
get_persistent_method(...), |
||||
this_ |
||||
); |
||||
``` |
||||
|
||||
`php::getCallableScope()` builds both the called scope and the real instance information from `this_`. All scoped calls within a method reference the same `tmp_var_1`, so repeated calls within a loop do not recreate the synthetic frame. |
||||
|
||||
If a method never uses scoped dynamic calls, first-class callables, or scoped callbacks, the compiler does not generate this variable. |
||||
|
||||
### 3.5 Usage Entry Points |
||||
|
||||
#### `php::callScoped()` |
||||
|
||||
Used for dynamic function or object method calls. Internally, `call_function_impl()` uses `CallableScope::resolve()` to obtain a `zend_fcall_info_cache`, then executes `zend_call_function()`. |
||||
|
||||
The typical scenario is when the compiler cannot resolve an object method into a Native Call, but still needs to preserve access to the current class's private/protected members. |
||||
|
||||
#### `php::makeScopedCallable()` |
||||
|
||||
Used for first-class callable syntax. The result of this syntax must be a real `Closure`, so even if the target method is public, it cannot simply return the original callback array or string. |
||||
|
||||
```php |
||||
$callback = self::privateMethod(...); |
||||
$callback = $this->publicMethod(...); |
||||
``` |
||||
|
||||
Ordinary methods create a Closure through `zend_create_fake_closure()`. If Zend returns `ZEND_ACC_CALL_VIA_TRAMPOLINE`, a forwarding Closure is used to preserve the dynamic semantics of magic `__call()` / `__callStatic()`. |
||||
|
||||
#### `php::prepareScopedCallback()` |
||||
|
||||
Used to pass a callback to PHP built-in functions such as `array_map()` and `usort()`. The goal here is only for the built-in function to invoke the callback correctly; it is not required that the argument itself become a Closure. |
||||
|
||||
Therefore, it first reuses the original value of the following callbacks: |
||||
|
||||
- Public methods; |
||||
- Located by absolute class name; |
||||
- Not relying on a trampoline. |
||||
|
||||
Only private/protected methods, `self` / `parent` / `static` relative callbacks, or trampolines create a Closure. This avoids unconditionally allocating a fake Closure each time a built-in function is called within a loop. |
||||
|
||||
### 3.6 Why `self` / `parent` / `static` Still Need Runtime Recognition |
||||
|
||||
In direct syntax, `self::class` can be expanded to a concrete class name at compile time, but PHP callbacks also allow dynamic values: |
||||
|
||||
```php |
||||
$class = 'self'; |
||||
$callback = [$class, 'method']; |
||||
``` |
||||
|
||||
In this case, only at runtime can we know whether the class name in the array is a relative class name. Therefore `isRelativeCallableClass()` cannot be fully moved to compile time. For known absolute public callbacks, this check returns false quickly and reuses the original value. |
||||
|
||||
## 4. `php::UserCodeScopeGuard` |
||||
|
||||
### 4.1 Responsibilities and Scope of Application |
||||
|
||||
`UserCodeScopeGuard` serves fully dynamic `call_user_func()` / `call_user_func_array()`, callback maps, and argument-unpacking scenarios where the compiler cannot statically rewrite the callback. |
||||
|
||||
```php |
||||
$args = [[$this, 'privateMethod'], 1]; |
||||
call_user_func(...$args); |
||||
``` |
||||
|
||||
A built-in function callback may be at a fixed position, a reverse position, in a named argument, or even a single function may have multiple callbacks. Before executing the `...$args` unpacking, the compiler does not know the final positional/named argument layout and cannot call `prepareScopedCallback()` only on the corresponding values. |
||||
|
||||
`call_user_func*` itself is a fully dynamic call boundary of the ZendVM; regardless of whether the callback appears explicitly, no fake Closure is created. If the callable array uses `self`, `parent`, or `static`, `normalizeCallableClass()` first converts the class part into a real class name: |
||||
|
||||
- `self` becomes `CallableScope::lexicalScope()`; |
||||
- `parent` becomes the parent class of the lexical scope; |
||||
- `static` becomes `CallableScope::calledScope()`. |
||||
|
||||
Normalization only copies the callback arrays that need modification. Absolute class names, object callbacks, Closures, and ordinary function names keep their original values. |
||||
|
||||
`preg_replace_callback_array()` is a special case of a callback map. Zend resolves callbacks in the map item by item internally; if the map were wrapped in advance, each call would perform an O(N) scan and might trigger array COW and multiple Closure allocations. Therefore the compiler keeps the original map and creates a `UserCodeScopeGuard` once at the method entry, letting Zend resolve directly with the correct scope. |
||||
|
||||
Apart from fully dynamic calls, callback maps, and argument unpacking, ordinary callback arguments must not use this guard; as long as the AST parameter position of a single callback is known, the `CallableScope` path should be used. |
||||
|
||||
### 4.2 Implementation |
||||
|
||||
The constructor walks upward from `EG(current_execute_data)` to find the nearest user-code frame, skipping internal frames: |
||||
|
||||
```cpp |
||||
while (frame && (!frame->func || !ZEND_USER_CODE(frame->func->type))) { |
||||
frame = frame->prev_execute_data; |
||||
} |
||||
``` |
||||
|
||||
Once found, it saves it and uses `CallableScope::lexicalScope()` to set the visibility scope: |
||||
|
||||
```cpp |
||||
function_ = frame->func; |
||||
previous_scope_ = function_->common.scope; |
||||
function_->common.scope = callable_scope.lexicalScope(); |
||||
``` |
||||
|
||||
The destructor restores `previous_scope_`. The class is non-copyable and non-movable, ensuring one construction corresponds to one restoration. If no user-code frame is available, it throws: |
||||
|
||||
```text |
||||
A user-code frame is required for scoped dynamic callback calls |
||||
``` |
||||
|
||||
This guard operates on the user-code frame found from the current request's execution chain, not on the persistent internal methods registered by TypePHP in MINIT. `EG(current_execute_data)` itself belongs to the current executor context. Its impact window is limited to the RAII lifecycle of the current AOT method call. |
||||
|
||||
### 4.3 Compiler Generation Pattern |
||||
|
||||
The compiler maintains a semantically clear flag: |
||||
|
||||
```php |
||||
FunctionContext::$needsUserCodeCallableScope |
||||
``` |
||||
|
||||
When the compiler encounters a dynamic callback of `call_user_func*`, or a built-in function known to synchronously invoke callbacks has an argument unpacking that cannot be matched, `markUserCodeCallableScope()` sets this flag. The state belongs to the current `FunctionContext`, so ordinary methods, nested Closures, and Fibers are independent and do not leak the guard into an outer function incorrectly. Each function body generates only one at its entry: |
||||
|
||||
```cpp |
||||
php::CallableScope tmp_var_1 = php::getCallableScope(..., this_); |
||||
php::UserCodeScopeGuard tmp_var_2{tmp_var_1}; |
||||
``` |
||||
|
||||
Even if the call form is `call_user_func($closure)`, and inside the Closure there is another call via |
||||
`call_user_func(['self', 'method'])`, each layer only reads its own |
||||
`FunctionContext`, lexical scope, and `$this`, and cannot reuse or pollute the outer guard. |
||||
|
||||
It is not created per call site or per loop iteration. Methods without the above dynamic callbacks incur no such cost. |
||||
|
||||
### 4.4 Why This Fallback Is Currently Kept |
||||
|
||||
If it were completely removed, the compiler would have to add a structured argument binding and rewriting flow after argument unpacking completes, correctly handling: |
||||
|
||||
- Merging positional and named arguments; |
||||
- Forward and reverse positions of callbacks; |
||||
- Multiple callbacks in one function; |
||||
- Callback maps; |
||||
- PHP error semantics when arguments are duplicated, missing, or overridden during unpacking. |
||||
|
||||
This is not a localized replacement, but a medium-scale refactoring of `parseCallArgs()` and the argument container generation flow. Until a unified runtime argument post-processing mechanism is completed, keeping the strictly controlled `UserCodeScopeGuard` is simpler and more reliable. |
||||
|
||||
## 5. `php::FakeScopeGuard` |
||||
|
||||
### 5.1 Responsibilities |
||||
|
||||
`FakeScopeGuard` is the RAII wrapper for `EG(fake_scope)`. Some Zend APIs do not accept an explicit call frame; instead, they directly read `EG(fake_scope)` to determine class member visibility or perform class-scope-related operations. Only these APIs should use it. |
||||
|
||||
Current typical scenarios include: |
||||
|
||||
- Dynamic property reads, writes, and property hooks; |
||||
- Zend object handler calls; |
||||
- Default values or object initialization under class scope; |
||||
- Zend operations related to exception objects; |
||||
- Other Zend internal interfaces that explicitly read `EG(fake_scope)`. |
||||
|
||||
TypePHP's property access generator passes the current fake scope to the PHPX property helper through `FakeScopeGuard::current()`. |
||||
|
||||
### 5.2 Implementation |
||||
|
||||
On construction it saves the old value and sets the new value; on destruction it restores it: |
||||
|
||||
```cpp |
||||
explicit FakeScopeGuard(Scope scope) noexcept : previous_(current()) { |
||||
EG(fake_scope) = scope; |
||||
} |
||||
|
||||
~FakeScopeGuard() noexcept { |
||||
restore(); |
||||
} |
||||
``` |
||||
|
||||
`Scope` is deduced through `decltype(EG(fake_scope))` to be compatible with both PHP 8.4's mutable pointer and PHP 8.5's pointer-to-const. `restore()` is idempotent and can be safely called once in advance. |
||||
|
||||
### 5.3 Zend Bailout Considerations |
||||
|
||||
C++ exception unwinding executes destructors, but Zend bailout uses `longjmp` and does not execute C++ destructors. If a guard's lifecycle crosses a bailout boundary, it must be explicitly invoked in the corresponding `zend_catch` path: |
||||
|
||||
```cpp |
||||
fake_scope_guard.restore(); |
||||
``` |
||||
|
||||
and then continue the bailout or convert the exception. Relying solely on the destructor to handle bailout is incorrect. |
||||
|
||||
### 5.4 Non-applicable Scenarios |
||||
|
||||
`FakeScopeGuard` cannot replace `CallableScope`: |
||||
|
||||
- It has no synthetic frame; |
||||
- It cannot carry `$this`; |
||||
- It cannot fully express lexical scope and called scope; |
||||
- The resolution semantics of `zend_is_callable_at_frame()` should not be indirectly simulated through a global fake scope. |
||||
|
||||
Likewise, `EG(fake_scope)` must not be unconditionally set at the entry of every AOT method just because "private access might be needed." This would widen the impact of global state and make unrelated native-intensive calls bear the cost. |
||||
|
||||
## 6. Call Flows of the Three Scopes |
||||
|
||||
### 6.1 Known Dynamic Method Call |
||||
|
||||
```text |
||||
AOT method entry |
||||
-> lazily generated CallableScope |
||||
-> php::callScoped() |
||||
-> CallableScope::resolve() |
||||
-> zend_is_callable_at_frame(synthetic frame) |
||||
-> zend_call_function() |
||||
``` |
||||
|
||||
The whole process does not modify the real Zend frame. |
||||
|
||||
### 6.2 Known Built-in Function Callback |
||||
|
||||
```text |
||||
compiler marks callback argument |
||||
-> prepareScopedCallback(value, CallableScope) |
||||
-> public absolute callback: reuse value |
||||
-> scoped/trampoline callback: create Closure |
||||
-> call PHP internal function |
||||
``` |
||||
|
||||
First-class callables use the same resolution basis but must call `makeScopedCallable()` and return a Closure. |
||||
|
||||
### 6.3 Callback Inside Argument Unpacking |
||||
|
||||
```text |
||||
AOT method entry |
||||
-> UserCodeScopeGuard changes nearest user-code frame scope |
||||
-> internal function receives expanded arguments |
||||
-> Zend resolves hidden callback using that frame scope |
||||
-> method exit / C++ exception unwind |
||||
-> guard restores original scope |
||||
``` |
||||
|
||||
### 6.4 Property or Object Handler |
||||
|
||||
```text |
||||
save EG(fake_scope) |
||||
-> install FakeScopeGuard |
||||
-> call Zend property/object API |
||||
-> restore on normal/C++ exception exit |
||||
-> explicitly restore in zend_catch if bailout is possible |
||||
``` |
||||
|
||||
## 7. Forbidden Mixing and Maintenance Constraints |
||||
|
||||
1. Do not use `FakeScopeGuard` to resolve callables. |
||||
2. Do not modify the real user-code frame for ordinary known callbacks; use `prepareScopedCallback()`. |
||||
3. Do not let `UserCodeScopeGuard` become the general entry point for all dynamic calls again. |
||||
4. Do not recreate `CallableScope` at call sites within loops; it should be hoisted to the method entry by `FunctionContext` and reused. |
||||
5. Do not cache the function, object, or synthetic frame borrowed by `CallableScope` beyond the request. |
||||
6. Do not change first-class callables to return the original callback; its PHP result type must be Closure. |
||||
7. When adding a PHP built-in function that synchronously invokes callbacks, update the callback argument description table, noting the position, argument name, and whether it is a callback map. |
||||
8. Functions that save a callback but do not invoke it immediately must not mark the scope fallback merely because they receive a callable, for example `spl_autoload_register()`. |
||||
9. When adding a `FakeScopeGuard` usage that crosses a Zend bailout, code review must check whether `zend_catch` explicitly restores it. |
||||
|
||||
## 8. Performance Model |
||||
|
||||
| Path | Main Cost | Optimization Strategy | |
||||
| --- | --- | --- | |
||||
| `CallableScope` | Initializing one synthetic frame | At most once per AOT method, reused across loops | |
||||
| `callScoped()` | Dynamic resolution by `zend_is_callable_at_frame()` | Used only for dynamic calls; resolvable Native Calls do not enter this path | |
||||
| `prepareScopedCallback()` | One callable resolution | Public absolute callbacks do not create a Closure | |
||||
| `makeScopedCallable()` | Callable resolution and Closure allocation | Used only for first-class callables | |
||||
| `UserCodeScopeGuard` | One pointer lookup and write at method entry, plus restoration at exit | Generated only for `call_user_func*`, callback maps, or unresolved unpack callbacks | |
||||
| `FakeScopeGuard` | Two executor-global pointer assignments | Only surrounds Zend APIs that actually read fake scope | |
||||
|
||||
This design deliberately keeps common pure Native Calls, callback-free methods, and public callbacks on the shortest path. Do not sink low-frequency fallbacks into every call just to unify the surface form. |
||||
|
||||
## 9. Test Requirements |
||||
|
||||
Scope changes should cover at least the following layers: |
||||
|
||||
- PHPX unit tests: `FakeScopeGuard` save, nesting, restoration, and early `restore()`; |
||||
- Compiler structure tests: one method generates only one `php::getCallableScope()`, with multiple call sites reusing the same variable; |
||||
- PHPT: private/protected callbacks, non-static `self::method(...)`, public callbacks; |
||||
- PHPT: a mix of public and scoped callbacks in a callback map; |
||||
- PHPT: a private callback inside `...$args` is callable, and the scope is restored after an exception exit; |
||||
- PHPT: scope generation paths in Closure, Fiber, and ordinary methods; |
||||
- Regression tests: pure Native Calls must not generate extra Scope guards. |
||||
|
||||
Current related tests include: |
||||
|
||||
- `phpunit/src/ScopedCallContextTest.php` |
||||
- `phpunit/code/scoped-call-context-reuse.php` |
||||
- `tests/compiler/place-holder/non-static-self.phpt` |
||||
- `tests/compiler/callable/scoped-internal-callbacks.phpt` |
||||
- `tests/compiler/callable/unpacked-callback-scope-restored.phpt` |
||||
- PHPX `tests/src/scope_guard.cpp` |
||||
|
||||
PHPT tests involving dynamic calls that throw exceptions may trigger known ZendVM memory leak reports; only when the leak is confirmed to come from Zend's dynamic call exception path may the test locally set `USE_ZEND_ALLOC=0`, and memory checking must not be disabled globally. |
||||
|
||||
## 10. Code Location Index |
||||
|
||||
| Content | Location | |
||||
| --- | --- | |
||||
| `CallableScope` and public helper declarations | `vendor/swoole/phpx/include/phpx.h` | |
||||
| Callable resolution and wrapping | `vendor/swoole/phpx/src/core/base.cc`, `vendor/swoole/phpx/src/core/closure.cc` | |
||||
| `FakeScopeGuard` | `vendor/swoole/phpx/include/phpx_fake_scope_guard.h` | |
||||
| `UserCodeScopeGuard` | `vendor/swoole/phpx/include/typephp_helper.h`, `src/core/scope.cc` | |
||||
| `php::getCallableScope()` | `vendor/swoole/phpx/include/typephp_helper.h` | |
||||
| Callback marking and Scope variable generation | `src/CompilerBase.php` | |
||||
| Callback argument wrapping | `src/Generator/CallArgumentGenerator.php` | |
||||
| Closure/Fiber fallback guard | `src/Generator/ClosureGenerator.php`, `FiberGenerator.php` | |
||||
| Method fallback guard | `src/Translator.php` | |
||||
| Scope state | `src/Context/FunctionContext.php` | |
||||
| Fake scope in property access | `src/Parser/PropertyAccessTrait.php` | |
||||
|
||||
## 11. Future Evolution Principles |
||||
|
||||
`UserCodeScopeGuard` is a long-term retained mechanism for complex dynamic calls and is not targeted for removal. It modifies the user-code frame in the current thread and current request, and restores it via RAII; under ZTS, different threads have their own execution contexts, so the modified frame state is not shared. |
||||
|
||||
`CallableScope` is used for the single scenario where the compiler can determine the callback location and call boundary, in order to reduce frame modification and Closure wrapping; it is a faster, more explicit path, not one required to cover all scenarios such as unpacking and multi-layer dynamic callbacks. When encountering combinations that are hard to prove safe statically, prefer keeping `UserCodeScopeGuard`, and do not forcibly rewrite it to `CallableScope` for the sake of formal uniformity. |
||||
|
||||
Before adding a new Scope abstraction in the future, first confirm whether the Zend API depends on a synthetic call frame, a real user-code frame, or `EG(fake_scope)`. The name and type should directly express the managed Zend state, avoiding the reappearance of an overly broad general-purpose `Scope` class. |
||||
@ -0,0 +1,607 @@ |
||||
# Swoole AOT Strongly-Typed High-Performance Containers — Array Access Performance Improved by 10x |
||||
|
||||
> Std Container uses a PHPX Box to hold a concrete C++ template instance. For its storage |
||||
> and passing boundary relative to ordinary Zend Objects and Native |
||||
> Class Objects, see |
||||
> [OBJECT_STORAGE_AND_PASSING_MODELS.md](OBJECT_STORAGE_AND_PASSING_MODELS.md). |
||||
|
||||
The Swoole AOT compiler provides PHP with a set of `std` strongly-typed containers for replacing PHP Arrays in some performance-sensitive paths under AOT compile scenarios. They keep access syntax close to PHP while letting the compiler obtain a definite element type, key type, and container structure, thereby generating more direct, lower-overhead C++ code. |
||||
|
||||
## The Problem with PHP Arrays |
||||
|
||||
PHP Array is a very flexible data structure that can serve as a list, a hash table, a dictionary, or a struct: |
||||
|
||||
```php |
||||
$data = []; |
||||
$data[] = 1; |
||||
$data["name"] = "swoole"; |
||||
$data[10] = new stdClass(); |
||||
``` |
||||
|
||||
This flexibility brings convenience, but it also causes problems in large-scale projects and high-performance scenarios. |
||||
|
||||
### Programming-Convention Problems |
||||
|
||||
The key and value types of a PHP Array are not fixed, which easily leads to implicit conventions: |
||||
|
||||
```php |
||||
$user = [ |
||||
"id" => 1, |
||||
"name" => "alice", |
||||
"tags" => ["php", "swoole"], |
||||
]; |
||||
``` |
||||
|
||||
Structures like this usually rely on comments, documentation, or team conventions to guarantee correctness: |
||||
|
||||
```php |
||||
/** |
||||
* @param array{id:int, name:string, tags:string[]} $user |
||||
*/ |
||||
function saveUser(array $user): void |
||||
{ |
||||
} |
||||
``` |
||||
|
||||
But the runtime does not naturally guarantee: |
||||
|
||||
- `id` is always an int |
||||
- `name` always exists |
||||
- `tags` is always an array of strings |
||||
- whether the array is contiguous |
||||
- whether the key is int or string |
||||
- whether values of other types are mixed in |
||||
|
||||
This leads to a large amount of defensive code: |
||||
|
||||
```php |
||||
if (!isset($user["id"]) || !is_int($user["id"])) { |
||||
throw new InvalidArgumentException("invalid user id"); |
||||
} |
||||
``` |
||||
|
||||
In AOT compile scenarios, uncertain types also limit compiler optimization. When the compiler cannot reliably infer the element types inside an array, it can only conservatively generate generic `php::Array` / `php::Var` operations. |
||||
|
||||
### Performance Problems |
||||
|
||||
PHP Array is a generic HashTable suited to dynamic-language semantics, but it is not the optimal data structure for all scenarios. |
||||
|
||||
Typical overheads include: |
||||
|
||||
- each element needs to store zval type information |
||||
- key/value are both dynamic structures |
||||
- mixing int keys and string keys requires compatibility handling |
||||
- element access usually requires hash lookup or indirect access |
||||
- the memory layout is not contiguous, resulting in a lower CPU cache hit rate |
||||
- the value type is uncertain, possibly requiring dynamic type conversion before computation |
||||
- mechanisms such as copy-on-write and reference counting add extra runtime cost |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
$sum = 0; |
||||
foreach ($numbers as $n) { |
||||
$sum += $n; |
||||
} |
||||
``` |
||||
|
||||
If `$numbers` is an ordinary PHP Array, the compiler cannot confirm that every element is definitely an int. Even if it is known from the business logic to be `int[]`, the underlying code still needs to keep dynamic type handling capability. |
||||
|
||||
## std Strongly-Typed Containers |
||||
|
||||
Swoole AOT provides `std` containers to express "the structure and element type of this container are definite at compile time." |
||||
|
||||
Currently supported: |
||||
|
||||
- `std::array` |
||||
- `std::vector` |
||||
- `std::ordered_map` |
||||
- `std::map` |
||||
|
||||
Their goal is not to fully replace PHP Array, but to be used in performance-sensitive, structurally stable, and clearly-typed code paths. |
||||
|
||||
## std::array |
||||
|
||||
`std::array` is a fixed-length array whose length and element type are determined at compile time. |
||||
|
||||
```php |
||||
function main(): void |
||||
{ |
||||
$array = std::array(Type::Int, 100); |
||||
|
||||
$array[0] = 123; |
||||
$array[99] = 456; |
||||
|
||||
var_dump($array[0]); |
||||
} |
||||
``` |
||||
|
||||
Characteristics: |
||||
|
||||
- fixed length |
||||
- supports bounds checking |
||||
- fixed element type |
||||
- supports nested structures |
||||
- suited for matrices, fixed-length buffers, and fixed-structure data |
||||
|
||||
Nested example: |
||||
|
||||
```php |
||||
function main(): void |
||||
{ |
||||
$matrix = std::array( |
||||
std::array(Type::Int, 4), |
||||
3 |
||||
); |
||||
|
||||
$matrix[0][0] = 10; |
||||
$matrix[2][3] = 99; |
||||
|
||||
var_dump($matrix[2][3]); |
||||
} |
||||
``` |
||||
|
||||
`std::array` supports copy of the same type: |
||||
|
||||
```php |
||||
function main(): void |
||||
{ |
||||
$a = std::array(Type::Int, 3); |
||||
$b = std::array(std::array(Type::Int, 3), 2); |
||||
|
||||
$b[1][0] = 10; |
||||
$b[1][1] = 20; |
||||
$b[1][2] = 30; |
||||
|
||||
$a = $b[1]; // allowed, types are exactly identical, performs a std::array copy |
||||
} |
||||
``` |
||||
|
||||
## std::vector |
||||
|
||||
`std::vector` is a dynamically-sized contiguous array. |
||||
|
||||
```php |
||||
function main(): void |
||||
{ |
||||
$vector = std::vector(Type::Int); |
||||
|
||||
$vector[] = 1; |
||||
$vector[] = 2; |
||||
$vector[] = 3; |
||||
|
||||
var_dump($vector[1]); |
||||
var_dump(count($vector)); |
||||
} |
||||
``` |
||||
|
||||
An initial length can also be specified: |
||||
|
||||
```php |
||||
$vector = std::vector(Type::Float, 1024); |
||||
``` |
||||
|
||||
Characteristics: |
||||
|
||||
- dynamic length |
||||
- contiguous memory |
||||
- suited for a large number of elements of the same type |
||||
- better access performance than PHP Array |
||||
- fixed element type |
||||
|
||||
Vectors of the same type can be copied: |
||||
|
||||
```php |
||||
$a = std::vector(Type::Int); |
||||
$b = std::vector(Type::Int); |
||||
|
||||
$b[] = 10; |
||||
$b[] = 20; |
||||
|
||||
$a = $b; // allowed, types are exactly identical, performs a container copy |
||||
``` |
||||
|
||||
### Modifying Elements in foreach |
||||
|
||||
When iterating over `std::vector`, `std::map`, or `std::ordered_map`, you can update the values of existing elements, for example using `+=`: |
||||
|
||||
```php |
||||
foreach ($vector as $index => $value) { |
||||
$vector[$index] += 10; |
||||
} |
||||
``` |
||||
|
||||
During iteration you cannot perform structural modifications that may invalidate the C++ iterator, including appending elements, inserting or overwriting keys, `unset()`, and replacing the container as a whole. The compiler reports these directly as errors. When the structure needs to change, record the keys to be processed first and apply the modifications uniformly after the `foreach` ends. |
||||
|
||||
## std::ordered_map |
||||
|
||||
`std::ordered_map` is an ordered key-value container. |
||||
|
||||
```php |
||||
function main(): void |
||||
{ |
||||
$map = std::ordered_map( |
||||
Type::String, |
||||
Type::Int |
||||
); |
||||
|
||||
$map["a"] = 1; |
||||
$map["b"] = 2; |
||||
|
||||
var_dump($map["a"]); |
||||
} |
||||
``` |
||||
|
||||
Characteristics: |
||||
|
||||
- fixed key type |
||||
- fixed value type |
||||
- suited for scenarios requiring a stable key-value structure |
||||
- supports string keys and int keys |
||||
|
||||
Example: |
||||
|
||||
```php |
||||
$map = std::ordered_map(Type::Int, Type::Float); |
||||
|
||||
$map[10] = 1.25; |
||||
$map[20] = 3.5; |
||||
``` |
||||
|
||||
ordered_map of the same type can be copied: |
||||
|
||||
```php |
||||
$a = std::ordered_map(Type::Int, Type::Int); |
||||
$b = std::ordered_map(Type::Int, Type::Int); |
||||
|
||||
$b[10] = 100; |
||||
$a = $b; |
||||
``` |
||||
|
||||
## std::map |
||||
|
||||
`std::map` is a hash-table key-value container. |
||||
|
||||
```php |
||||
function main(): void |
||||
{ |
||||
$map = std::map( |
||||
Type::Int, |
||||
Type::Int |
||||
); |
||||
|
||||
$map[100] = 1; |
||||
$map[200] = 2; |
||||
|
||||
var_dump($map[100]); |
||||
} |
||||
``` |
||||
|
||||
Characteristics: |
||||
|
||||
- fixed key type |
||||
- fixed value type |
||||
- suited for a large number of key-value lookups |
||||
- usually used for mapping scenarios that do not require ordering |
||||
|
||||
map of the same type can be copied: |
||||
|
||||
```php |
||||
$a = std::map(Type::Int, Type::Int); |
||||
$b = std::map(Type::Int, Type::Int); |
||||
|
||||
$b[1] = 42; |
||||
$a = $b; |
||||
``` |
||||
|
||||
## Supported Element Types |
||||
|
||||
Type symbols: |
||||
|
||||
```php |
||||
Type::Int |
||||
Type::Float |
||||
Type::Bool |
||||
Type::String |
||||
Type::Array |
||||
Type::Object |
||||
Type::Any |
||||
Type::Stream |
||||
Type::Box |
||||
``` |
||||
|
||||
Class names can also be used as the value type: |
||||
|
||||
```php |
||||
class User |
||||
{ |
||||
} |
||||
|
||||
$vector = std::vector(User::class); |
||||
$array = std::array(User::class, 10); |
||||
$map = std::ordered_map(Type::String, User::class); |
||||
``` |
||||
|
||||
Class-typed containers check the object type at write time to prevent mixing in incorrect objects. |
||||
|
||||
## Conversion with PHP Array |
||||
|
||||
When a std container is assigned to an ordinary variable, it is automatically converted to a PHP Array: |
||||
|
||||
```php |
||||
function main(): void |
||||
{ |
||||
$vector = std::vector(Type::Int); |
||||
$vector[] = 1; |
||||
$vector[] = 2; |
||||
|
||||
$array = $vector; // converted to PHP Array |
||||
|
||||
var_dump(is_array($array)); // true |
||||
} |
||||
``` |
||||
|
||||
If the lvalue itself is a std container of the same type, a container copy is performed instead of converting to a PHP Array: |
||||
|
||||
```php |
||||
$a = std::vector(Type::Int); |
||||
$b = std::vector(Type::Int); |
||||
|
||||
$a = $b; // std::vector copy |
||||
``` |
||||
|
||||
If the types differ, the copy is not allowed: |
||||
|
||||
```php |
||||
$a = std::vector(Type::Int); |
||||
$b = std::vector(Type::Float); |
||||
|
||||
$a = $b; // compile failure |
||||
``` |
||||
|
||||
## UnsafePtr and native Function Parameters |
||||
|
||||
For scenarios where a std container reference needs to be passed between native functions, the `UnsafePtr` parameter can be used. |
||||
|
||||
The caller does not need to explicitly create an unsafe pointer: |
||||
|
||||
```php |
||||
function update(UnsafePtr $ptr): void |
||||
{ |
||||
$vector = std::unsafe_cast( |
||||
std::vector(Type::Int), |
||||
$ptr |
||||
); |
||||
|
||||
$vector[0] = 100; |
||||
} |
||||
|
||||
function main(): void |
||||
{ |
||||
$vector = std::vector(Type::Int, 1); |
||||
update($vector); // the compiler automatically converts to an UnsafePtr box |
||||
} |
||||
``` |
||||
|
||||
Rules: |
||||
|
||||
- `UnsafePtr` can only be used as a native function or method parameter |
||||
- the argument must be a std container variable |
||||
- local variables are never `UnsafePtr` |
||||
- the second argument of `std::unsafe_cast()` must be an `UnsafePtr` parameter in the current function signature |
||||
- the runtime validates the container type ID; inconsistent types throw an exception |
||||
|
||||
This ensures that unsafe casts do not depend on user-written temporary code and also avoids propagating unsafe pointers in local variables. |
||||
|
||||
## A Brief Introduction to the Compilation Principle |
||||
|
||||
An ordinary PHP Array is usually represented as a dynamic structure during AOT compilation: |
||||
|
||||
```cpp |
||||
php::Array |
||||
php::Var |
||||
``` |
||||
|
||||
This means every access needs to preserve PHP's dynamic semantics. |
||||
|
||||
std containers are different. The compiler records container metadata while parsing the code: |
||||
|
||||
- container type |
||||
- element type |
||||
- key type |
||||
- class type |
||||
- dimension information of `std::array` |
||||
- type ID |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
$vector = std::vector(Type::Int); |
||||
``` |
||||
|
||||
can generate something like: |
||||
|
||||
```cpp |
||||
php::StdVector<php::Int> vector; |
||||
``` |
||||
|
||||
For another example: |
||||
|
||||
```php |
||||
$array = std::array(std::array(Type::Int, 3), 2); |
||||
``` |
||||
|
||||
can generate something like: |
||||
|
||||
```cpp |
||||
php::StdArray<php::StdArray<php::Int, 3>, 2> array; |
||||
``` |
||||
|
||||
Therefore the compiler can directly generate strongly-typed access code: |
||||
|
||||
```php |
||||
$array[1][2] = 100; |
||||
``` |
||||
|
||||
which corresponds approximately to: |
||||
|
||||
```cpp |
||||
array[safeIndex(1, 2)][safeIndex(2, 3)] = 100; |
||||
``` |
||||
|
||||
This brings several benefits: |
||||
|
||||
- type conversion is determined at compile time |
||||
- shorter container access paths |
||||
- more compact element layout |
||||
- the C++ compiler can further optimize |
||||
- errors surface earlier at compile time |
||||
- friendlier to performance-sensitive code |
||||
|
||||
## Usage Recommendations |
||||
|
||||
Scenarios suitable for std containers: |
||||
|
||||
- large-scale numeric computation |
||||
- fixed-structure data |
||||
- a large number of elements of the same type |
||||
- array access in high-frequency loops |
||||
- mapping tables with stable key/value types |
||||
- hot paths that need to reduce the dynamic overhead of PHP Array |
||||
|
||||
Scenarios not suitable for std containers: |
||||
|
||||
- highly dynamic data structures |
||||
- frequently changing key/value types |
||||
- requiring full compatibility with PHP Array behavior |
||||
- flexible object structures at the business layer |
||||
- data from external input with an unstable structure |
||||
|
||||
The recommended approach is: keep using PHP Array or objects at the business boundary, and use std containers inside performance hot spots. |
||||
|
||||
## Performance Tests |
||||
### PHP Array |
||||
Test code: |
||||
```php |
||||
$u = (int)$argv[1]; |
||||
echo "u: $u\n"; |
||||
$r = rand(0, 10000); |
||||
$a = array_fill(0, 10000, 0); |
||||
|
||||
$begin = microtime(true); |
||||
for ($i = 0; $i < 10000; $i++) { |
||||
for ($j = 0; $j < 100000; $j++) { |
||||
$a[$i] += $j % $u; |
||||
} |
||||
$a[$i] += $r; |
||||
} |
||||
|
||||
echo $a[$r] . "\n"; |
||||
$end = microtime(true); |
||||
echo "sec: " . ($end - $begin) . "\n"; |
||||
``` |
||||
Test result: |
||||
```bash |
||||
php examples/array-loop/jit.php 999999 |
||||
u: 999999 |
||||
4999953010 |
||||
sec: 67.638107061386108 |
||||
``` |
||||
|
||||
### std::array |
||||
Test code: |
||||
```php |
||||
use native_types; |
||||
|
||||
function main(int $argc, array $argv): void |
||||
{ |
||||
$u = (int)$argv[2]; |
||||
echo "u: $u\n"; |
||||
$r = rand(0, 10000); |
||||
$a = std::array(Type::Int, 10000); |
||||
|
||||
$begin = microtime(true); |
||||
for ($i = 0; $i < 10000; $i++) { |
||||
for ($j = 0; $j < 100000; $j++) { |
||||
$a[$i] += $j % $u; |
||||
} |
||||
$a[$i] += $r; |
||||
} |
||||
|
||||
echo $a[$r] . "\n"; |
||||
$end = microtime(true); |
||||
echo "sec: " . ($end - $begin) . "\n"; |
||||
} |
||||
``` |
||||
|
||||
Test result: |
||||
```shell |
||||
./main examples/array-loop/main.php 999999 |
||||
u: 999999 |
||||
4999950397 |
||||
sec: 6.3918659687042236 |
||||
``` |
||||
|
||||
### C++ Test |
||||
Test code: |
||||
```cpp |
||||
#include <iostream> |
||||
#include <vector> |
||||
#include <cstdlib> |
||||
#include <ctime> |
||||
#include <chrono> |
||||
|
||||
int main(int argc, char* argv[]) { |
||||
std::srand(static_cast<unsigned>(std::time(nullptr))); |
||||
|
||||
long u = std::stoi(argv[1]); |
||||
std::cout << "u: " << u << "\n"; |
||||
|
||||
long r = std::rand() % 10001; |
||||
std::vector<long> a(10000, 0); |
||||
|
||||
auto begin = std::chrono::high_resolution_clock::now(); |
||||
|
||||
for (int i = 0; i < 10000; i++) { |
||||
for (int j = 0; j < 100000; j++) { |
||||
a[i] += j % u; |
||||
} |
||||
a[i] += r; |
||||
} |
||||
|
||||
std::cout << a[r] << "\n"; |
||||
|
||||
auto end = std::chrono::high_resolution_clock::now(); |
||||
std::chrono::duration<double> diff = end - begin; |
||||
std::cout << "sec: " << diff.count() << "\n"; |
||||
|
||||
return 0; |
||||
} |
||||
``` |
||||
Test result |
||||
```bash |
||||
g++ examples/array-loop/loop.cc -o loop -O3 |
||||
./loop 999999 |
||||
u: 999999 |
||||
4999954742 |
||||
sec: 6.22351 |
||||
swoole@swoole-26:~/workspace/aot/compiler$ |
||||
``` |
||||
|
||||
### Conclusion |
||||
The `std::array` container provided by the `AOT` compiler is almost `10` times as fast as `PHP Array`, and its performance is completely consistent with C++'s `std::vector`. |
||||
|
||||
## Summary |
||||
|
||||
PHP Array is a general, flexible, and highly expressive data structure, but its dynamism brings costs in type specification and performance. |
||||
|
||||
Swoole AOT's std containers provide a path better suited for compiler optimization: |
||||
|
||||
- use `std::array` to express fixed-length strongly-typed arrays |
||||
- use `std::vector` to express dynamic contiguous strongly-typed arrays |
||||
- use `std::ordered_map` / `std::map` to express strongly-typed mappings |
||||
- an ordinary variable receiving a std container is automatically converted to a PHP Array |
||||
- std containers of the same type support native copy |
||||
- UnsafePtr supports safely passing container references between native functions |
||||
|
||||
They let PHP code maintain high readability while providing the AOT compiler with sufficiently clear type information, thereby achieving more stable and more predictable performance. |
||||
@ -0,0 +1,48 @@ |
||||
# Test Coverage Checklist |
||||
|
||||
`bin/analyze-test-coverage.php` generates a coverage checklist from the source of PHPT and compiler PHPUnit fixtures. It is a static test-intent analysis tool and does not replace test execution. |
||||
|
||||
## Usage |
||||
|
||||
```bash |
||||
# terminal summary |
||||
php bin/analyze-test-coverage.php |
||||
|
||||
# reviewable full matrix |
||||
php bin/analyze-test-coverage.php \ |
||||
--format=markdown \ |
||||
--output=build/test-coverage.md |
||||
|
||||
# for CI or other tools to read |
||||
php bin/analyze-test-coverage.php \ |
||||
--format=json \ |
||||
--output=build/test-coverage.json \ |
||||
--strict |
||||
``` |
||||
|
||||
By default it scans `tests/compiler`, `phpunit/src`, and `phpunit/code`. One or more PHPT files or directories can also be passed at the end of the command; `--no-phpunit` analyzes only PHPT, and `--php-versions=8.4,8.5` sets the PHP version columns of the matrix. |
||||
|
||||
`--strict` returns a non-zero status when there are unexpected source-parse failures or unresolvable PHPUnit fixture references. Samples in negative data providers that are intentionally unparseable by php-parser are recorded separately under `expected_parser_diagnostics` and are not disguised as tool failures. |
||||
|
||||
## Three categories of coverage evidence |
||||
|
||||
Each applicable `PHP version × feature` row records: |
||||
|
||||
- `positive_compile`: valid PHPT, or positive PHPUnit compile fixtures; |
||||
- `runtime_semantics`: valid PHPT containing `EXPECT`, `EXPECTF`, or `EXPECTREGEX`; |
||||
- `negative_diagnostic`: PHPT expecting diagnostics, or PHPUnit tests/data providers that explicitly expect failure. |
||||
|
||||
`XFAIL` and unconditional `SKIPIF` are not counted on any evidence axis. PHP version ranges are inferred from test titles, `PHP_VERSION_ID` conditions in `SKIPIF`, and version strings in PHPUnit data rows. |
||||
|
||||
## Denominator |
||||
|
||||
The report only gives ratios with an explicit denominator: |
||||
|
||||
- AST node coverage denominator: the concrete AST node kinds provided by the currently installed `nikic/php-parser`; `Expr_Error`, used for error recovery, is not counted. |
||||
- Feature-axis coverage denominator: the number of rows in the feature catalog with `introduced <= target PHP version`. Each of the positive-compile, runtime-semantics, and negative-diagnostic axes is computed independently. |
||||
|
||||
The tool does not combine the three axes of different meanings into a single "overall project coverage". The full JSON preserves the feature catalog, per-item evidence sources, matrix, AST node occurrence counts, parse issues, and exclusion reasons, for further inspection by CI. |
||||
|
||||
## Classification boundaries |
||||
|
||||
AST nodes are extracted automatically by the parser. Semantic features that cannot be distinguished by nodes alone (such as DNF occurrence positions, property hook variants, `exit(message: ...)`) are supplemented by the explicit feature catalog in the analyzer. When adding a new language feature, register its introduction version and detection rule at the same time to keep the version matrix's denominator explicit. |
||||
@ -0,0 +1,462 @@ |
||||
# TypePHP WASM Technical Plan and Implementation Roadmap |
||||
|
||||
> Status: WASI 0.2 Component and Chrome Worker prototypes implemented |
||||
> Research date: 2026-08-07 |
||||
> Current goal: WASI 0.2 (Preview 2), NTS, single-threaded; WASI 0.1 not supported |
||||
|
||||
## 1. Document Purpose |
||||
|
||||
This document records the technical decisions, functional boundaries, runtime architecture, primary risks, validation methods, and phased implementation plan for TypePHP's WebAssembly support. |
||||
|
||||
The implementation validation completed on 2026-08-07 has proven that a trimmed PHP 8.5, the PHPX core, TypePHP-generated code, GMP, MPFR, and mpdecimal can be statically linked into a single module via the WASI SDK and run in Wasmtime. See [Building a TypePHP WASI Program](WASI_BUILD.md) for the reproducible build procedure. The remainder of this document also retains the browser-stage design goals. |
||||
|
||||
## 2. Core Conclusions |
||||
|
||||
The first TypePHP WASM release adopts the following path: |
||||
|
||||
```text |
||||
PHP source code |
||||
-> TypePHP compiler |
||||
-> TypePHP-generated C++ |
||||
-> WASI SDK compilation and static linking |
||||
+ PHP NTS |
||||
+ PHPX |
||||
+ TypePHP runtime |
||||
+ GMP / MPFR / mpdecimal |
||||
+ PHP embed/WASI runtime |
||||
-> typephp.wasm (WASI 0.2 command component) |
||||
``` |
||||
|
||||
Specific decisions are as follows: |
||||
|
||||
1. The first release reuses the current C++/Zend backend; it does not directly generate WAT/WASM, nor does it reimplement the PHP runtime. |
||||
2. Use the WASI SDK `wasm32-wasip2` sysroot to generate the Component directly; Chrome uses Jco to transpile it to ESM, without maintaining a second Emscripten ABI. |
||||
3. PHP, PHPX, TypePHP-generated code, and the high-precision libraries are all statically linked into a single `.wasm` module. |
||||
4. Wasmtime and Chrome together provide CLI, stdio, exit, clocks, random, and a controlled filesystem; the Chrome host always runs inside a Worker. |
||||
5. Only PHP NTS is supported; threads are not supported. |
||||
6. Fiber and Generator are disabled. |
||||
7. C++ exceptions and the `setjmp/longjmp` required by Zend bailout must be supported. |
||||
8. Keep the PHP stream framework and local streams, and disable network transports and features that depend on OS process capabilities. |
||||
9. WordPress Playground and other PHP-WASM projects serve only as a source of patches and porting experience, not as a dependency or codebase for TypePHP. |
||||
|
||||
This document describes the shortest viable path. For the long-term backend-neutral approach, see [BACKEND_NEUTRAL_IR.md](BACKEND_NEUTRAL_IR.md). The WASI prototype proves that the TypePHP frontend and semantic layers do not need to be rewritten for WASM. |
||||
|
||||
## 3. Why Not Adopt WordPress Playground |
||||
|
||||
WordPress Playground is a mature browser-based WordPress product, but it is not a lightweight PHP-WASM porting layer. Its repository and build system simultaneously serve: |
||||
|
||||
- Multiple PHP versions and extension combinations; |
||||
- WordPress distributions and their assets; |
||||
- Browser, Web Worker, and Node.js runtimes; |
||||
- Virtual filesystems, mounting, and persistence; |
||||
- Network proxying and browser HTTP adaptation; |
||||
- NPM packages, a website, developer tooling, and integration tests; |
||||
- WordPress-specific APIs and product features. |
||||
|
||||
TypePHP cannot directly reuse the PHP-WASM binary published by Playground, because TypePHP needs to statically link PHPX, the compiled C++, and the high-precision libraries together. If TypePHP forked Playground, it would also be bound to Playground's monorepo, Node/NPM build, version matrix, and product release cycle. |
||||
|
||||
Therefore, the following principles are adopted: |
||||
|
||||
- Do not fork WordPress Playground; |
||||
- Do not make `@php-wasm/*` a runtime dependency of TypePHP; |
||||
- Do not copy its WordPress, network proxy, file sync, and UI layers; |
||||
- Only study the PHP configure parameters, php-src patches, Emscripten compatibility handling, and the minimal C API; |
||||
- All borrowed patches must be split apart, have their sources attributed, and be verified to still apply to TypePHP's pinned PHP/Emscripten versions. |
||||
|
||||
Projects such as `seanmorris/php-wasm` and `soyuka/php-wasm` follow the same principle: they may serve as build references and issue indexes, but they do not become TypePHP's base repository. |
||||
|
||||
## 4. Goals and Non-Goals |
||||
|
||||
### 4.1 Current WASI Goals |
||||
|
||||
- Load TypePHP compilation artifacts in WASI runtimes such as Wasmtime. |
||||
- Execute the statically compiled TypePHP application entry point. |
||||
- Preserve TypePHP's current primary language semantics based on Zend and PHPX. |
||||
- Correctly handle the PHP request lifecycle, C++ exceptions, and Zend bailout. |
||||
- Support the GMP, MPFR, and mpdecimal high-precision types. |
||||
- Support the WASI filesystem and the necessary local PHP streams. |
||||
- Return deterministic, testable errors for unsupported features, rather than link failures or runtime crashes. |
||||
- Make the build process reproducible, with pinned php-src, WASI SDK, and numeric library versions. |
||||
|
||||
### 4.2 First-Phase Non-Goals |
||||
|
||||
- Direct browser execution without host adaptation. |
||||
- pthread, Web Worker-parallel PHP, or shared memory. |
||||
- Fiber and Generator. |
||||
- Dynamic extension loading. |
||||
- Runtime compilation of PHP source code or general-purpose `eval()`. |
||||
- TCP, UDP, Unix sockets, and listening ports. |
||||
- Network clients such as MySQL, PostgreSQL, and Redis. |
||||
- Network protocol implementations such as `curl`, FTP, and SMTP. |
||||
- `fork`, `exec`, `system`, `shell_exec`, `proc_open`, and signal handling. |
||||
- FFI, JIT, opcache, and the debugger. |
||||
- Full WordPress compatibility. |
||||
- Implementing asynchronous host calls in the first phase. |
||||
|
||||
## 5. Target Platform Selection |
||||
|
||||
### 5.1 Currently Using the WASI SDK |
||||
|
||||
For now, establish a command-line-verifiable baseline first. The WASI SDK has already been verified to provide, simultaneously: |
||||
|
||||
- A complete C/C++ to WebAssembly toolchain; |
||||
- Standard Wasm C++ exception handling; |
||||
- The SJLJ required by Zend bailout; |
||||
- A capability-based filesystem; |
||||
- libc, time, and random number interfaces. |
||||
|
||||
PHP, PHPX, and all TypePHP C++ translation units must use consistent Wasm EH/SJLJ parameters. The linker must treat inconsistent function signatures as fatal errors. |
||||
|
||||
### 5.2 Chrome Component Host |
||||
|
||||
Chrome currently cannot natively instantiate a Component. The builder uses Jco to transpile the same WASI 0.2 Component into core Wasm and ESM, and `examples/wasm-hello/typephp-worker.mjs` demonstrates the host entry point. The browser adaptation does not include PHP, PHPX, or high-precision type semantics. |
||||
|
||||
## 6. Artifacts and Runtime Model |
||||
|
||||
### 6.1 Release Artifacts |
||||
|
||||
The recommended minimal release artifacts are: |
||||
|
||||
```text |
||||
dist/ |
||||
├── typephp.wasm |
||||
└── typephp-wasm.mjs |
||||
``` |
||||
|
||||
All C/C++ code goes into `typephp.wasm`. The browser does not automatically provide WASI imports on its own; `typephp-wasm.mjs` serves as the loading entry point for the WASI host/adapter, responsible only for: |
||||
|
||||
- Fetching and instantiating the `.wasm`; |
||||
- Providing stdout/stderr; |
||||
- Initializing the in-memory filesystem; |
||||
- Implementing or wiring in host capabilities such as WASI clocks and random numbers; |
||||
- Invoking the exported TypePHP lifecycle interfaces; |
||||
- Converting status codes and error messages into JavaScript results. |
||||
|
||||
PHP semantics, Zend object operations, or TypePHP business logic should not be placed in the JavaScript loader. |
||||
|
||||
### 6.2 Lifecycle |
||||
|
||||
The recommended model is "module startup once, request repeatable": |
||||
|
||||
```text |
||||
instantiate wasm |
||||
-> typephp_wasm_module_startup() |
||||
-> typephp_wasm_request_startup() |
||||
-> TypePHP AOT entry |
||||
-> typephp_wasm_request_shutdown() |
||||
-> request can be executed again |
||||
-> typephp_wasm_module_shutdown() |
||||
``` |
||||
|
||||
Each request must have an independent PHP request memory pool. Successful execution, PHP exceptions, C++ exceptions, and Zend bailout must all enter a unified cleanup path. |
||||
|
||||
The module-exported API can start from the following minimal set; the names are subject to the actual implementation: |
||||
|
||||
```c |
||||
int typephp_wasm_module_startup(void); |
||||
int typephp_wasm_run(int argc, const char **argv); |
||||
const char *typephp_wasm_last_error(void); |
||||
void typephp_wasm_module_shutdown(void); |
||||
``` |
||||
|
||||
`typephp_wasm_run()` executes the statically linked AOT entry point; it is not responsible for parsing and compiling arbitrary PHP source code at runtime. |
||||
|
||||
## 7. PHP Build Strategy |
||||
|
||||
### 7.1 Base Configuration |
||||
|
||||
- Pin a specific php-src commit, rather than pinning only a branch name. |
||||
- NTS build. |
||||
- Disable existing SAPIs such as CLI, CGI, FPM, and Apache. |
||||
- Add a minimal `typephp_wasm` SAPI, or first validate the lifecycle with a minimal embed prototype before converging on a dedicated SAPI. |
||||
- Disable opcache/JIT. |
||||
- Statically link all extensions. |
||||
- Disable unneeded extensions and auto-detection to prevent the host environment from changing build results. |
||||
- Use `config.site` and a separate patch directory to record cross-compilation conclusions. |
||||
|
||||
For the first phase, do not directly copy other projects' complete configure parameters. Start from a minimal PHP core, and add extensions one by one according to TypePHP PHPT and runtime dependencies. |
||||
|
||||
### 7.2 Extension Layering |
||||
|
||||
It is recommended to divide extensions into three groups: |
||||
|
||||
1. **Must enable**: core, standard, SPL, date, pcre, hash, json, etc., required for TypePHP and Zend basic operation; the final set is subject to actual linking and test results. |
||||
2. **Optional local extensions**: ctype, filter, mbstring, tokenizer, fileinfo, zlib, etc., with no OS network dependencies, but they increase size. |
||||
3. **Disable in first phase**: sockets, curl, mysqli, PDO network drivers, pcntl, posix, FFI, shm, sysv, readline, opcache/JIT, etc. |
||||
|
||||
GMP, MPFR, and mpdecimal are first handled as static dependencies of the PHPX/TypePHP high-precision implementation; enabling PHP `ext/gmp` is not required. |
||||
|
||||
## 8. PHP Streams and OS Capabilities |
||||
|
||||
### 8.1 Do Not Disable the Entire Stream Subsystem |
||||
|
||||
The PHP standard library depends heavily on streams. Completely disabling streams would break file reads and writes, `php://`, include path handling, and some standard extensions, with little benefit and high compatibility cost. |
||||
|
||||
Keep in the first phase: |
||||
|
||||
- Ordinary file streams, backed by Emscripten MEMFS; |
||||
- `php://memory`; |
||||
- `php://temp`; |
||||
- Host mapping for `php://stdin`, `php://stdout`, and `php://stderr`; |
||||
- Whether `data://` is enabled is decided by size and security evaluation; |
||||
- Pure in-memory stream filters can be enabled as needed. |
||||
|
||||
### 8.2 Disable Network Streams |
||||
|
||||
The following should be disabled or not registered during the PHP build and runtime registration phases: |
||||
|
||||
- TCP, UDP, and Unix socket transports; |
||||
- The socket extension; |
||||
- Network-dependent wrappers such as `http://`, `https://`, and `ftp://`; |
||||
- `fsockopen()`, `pfsockopen()`, `stream_socket_*()`; |
||||
- Network database and network client extensions. |
||||
|
||||
In the first phase, PHP sockets should not be emulated via synchronous XHR or implicit JavaScript fetch. If HTTP is needed in the future, an explicit, authorizable asynchronous host API should be designed, rather than faking POSIX sockets. |
||||
|
||||
### 8.3 Other OS-Related Features |
||||
|
||||
The following capabilities must be disabled, degraded, or injected by the host: |
||||
|
||||
| Capability | First-phase strategy | |
||||
|---|---| |
||||
| Filesystem | MEMFS; optional read-only preloaded files | |
||||
| Current directory and paths | Virtual root directory; must not leak host paths | |
||||
| Environment variables | Loader-injected whitelist | |
||||
| Time | WASI clocks; the browser host implements this interface using browser clocks | |
||||
| Random numbers | WASI random; the browser host implements it using a secure random source, not a weak pseudo-random substitute | |
||||
| DNS, sockets | Not supported | |
||||
| Processes, shell | Not supported | |
||||
| Signals | Not supported | |
||||
| Users, groups, permissions | Fixed values or explicit errors | |
||||
| File locks | Cross-instance locks not supported in first phase; degrade as needed within a single instance | |
||||
| Persistence | Disabled by default; Chrome may explicitly enable OPFS filesystem snapshots | |
||||
|
||||
The compiler should progressively add WASM target capability checks: statically identifiable unsupported functions error at compile time; when a dynamic call cannot be statically determined, the runtime returns a deterministic error. These calls must never manifest as link-time missing symbols, empty functions, or undefined behavior. |
||||
|
||||
## 9. Exceptions, Bailout, and Cleanup |
||||
|
||||
This is the project's primary technical risk and must be validated before the full PHP feature port. |
||||
|
||||
### 9.1 Compilation Options |
||||
|
||||
When using native WebAssembly exceptions, C and C++ must use consistent `setjmp/longjmp` modes. The prototype is recommended to validate the following combination: |
||||
|
||||
```text |
||||
C compilation: |
||||
-sSUPPORT_LONGJMP=wasm |
||||
|
||||
C++ compilation: |
||||
-fwasm-exceptions |
||||
-sSUPPORT_LONGJMP=wasm |
||||
|
||||
Final link: |
||||
-fwasm-exceptions |
||||
-sSUPPORT_LONGJMP=wasm |
||||
``` |
||||
|
||||
All PHP, PHPX, TypePHP, and third-party C/C++ objects must use the same ABI and exception configuration. C++ exception catching cannot be enabled only at the final link stage. |
||||
|
||||
If target browser compatibility does not allow native Wasm EH, the Emscripten JavaScript exception mode can be researched as a fallback, but the two models must not be mixed in the same release. |
||||
|
||||
### 9.2 Boundary Rules |
||||
|
||||
- C++ exceptions must not cross exported functions unhandled into JavaScript. |
||||
- Zend bailout must be caught at the request top level and enter request shutdown. |
||||
- After bailout, dangling PHPX objects that depend on the destroyed request memory pool must not be destructed. |
||||
- PHPX `Variant`, `Object`, `Array`, and high-precision objects on the stack must complete destruction while the memory pool is still valid, or be taken over by a dedicated bailout-safe boundary. |
||||
- After one request fails, the next request must still be executable; otherwise the runtime can only be defined as a one-shot instance, which must be made explicit in the API. |
||||
|
||||
### 9.3 Must-Test Scenarios |
||||
|
||||
- PHP returns normally. |
||||
- PHP `throw` is caught by TypePHP code. |
||||
- An uncaught PHP exception reaches the request top level. |
||||
- `fatalError`/Zend bailout. |
||||
- C++ `throw` and `catch`. |
||||
- An exception is thrown when PHP calls C++ and C++ calls PHP again. |
||||
- PHPX objects and high-precision objects exist on the stack when bailout occurs. |
||||
- Execute success, failure, success three requests in sequence. |
||||
- Execute a request again after memory growth. |
||||
|
||||
## 10. Memory and High-Precision Libraries |
||||
|
||||
### 10.1 WASM Memory |
||||
|
||||
Use a single linear memory in the first phase, and validate `-sALLOW_MEMORY_GROWTH`. Record: |
||||
|
||||
- Initial memory; |
||||
- Maximum memory; |
||||
- PHP memory_limit; |
||||
- Zend memory reclamation after request end; |
||||
- Actual peak of the Emscripten allocator; |
||||
- Whether memory continues to grow after multiple requests. |
||||
|
||||
Do not choose `emmalloc` before benchmarking. PHP, GMP, MPFR, and mpdecimal are all allocation-intensive components; test size and runtime among candidates such as `dlmalloc` and `emmalloc`. |
||||
|
||||
### 10.2 GMP, MPFR, and mpdecimal |
||||
|
||||
- Statically compile all of them with the Emscripten toolchain. |
||||
- Disable assembly and host-CPU-specific optimizations. |
||||
- Pin limb, integer width, and ABI detection results. |
||||
- Do not depend on runtime dynamic library searching. |
||||
- Run existing BigInt, BigFloat, and Decimal PHPT, and add tests for maximum memory, division by zero, precision, rounding, and exception paths. |
||||
- Verify that library exceptions or allocation failures do not bypass PHP request cleanup. |
||||
|
||||
## 11. Recommended Repository Structure |
||||
|
||||
It is recommended to add a separate directory in the implementation phase, rather than scattering Emscripten conditionals into the existing build code: |
||||
|
||||
```text |
||||
wasm/ |
||||
├── README.md |
||||
├── build.sh |
||||
├── versions.env |
||||
├── config.site |
||||
├── cmake/ |
||||
│ └── TypePhpWasmToolchain.cmake |
||||
├── patches/ |
||||
│ ├── php-src/ |
||||
│ ├── gmp/ |
||||
│ ├── mpfr/ |
||||
│ └── mpdecimal/ |
||||
├── sapi/ |
||||
│ └── typephp_wasm/ |
||||
├── runtime/ |
||||
│ └── typephp-wasm.mjs |
||||
└── tests/ |
||||
``` |
||||
|
||||
Maintenance principles: |
||||
|
||||
- Patches should be small and independent, one patch per compatibility issue; |
||||
- Each patch records its upstream version, source, reason, and removal condition; |
||||
- Download caches are not committed to Git; |
||||
- php-src, Emscripten, and third-party libraries are locked by checksums; |
||||
- Build artifacts do not enter the source repository; |
||||
- CI keeps at least debug and release builds. |
||||
|
||||
## 12. Phased Implementation Plan |
||||
|
||||
### Phase 0: Toolchain Risk Validation |
||||
|
||||
Goal: Prove that the critical low-level mechanisms are feasible before integrating the full TypePHP. |
||||
|
||||
- Pin the Emscripten version. |
||||
- Compile a minimal mixed C/C++ program. |
||||
- Validate C++ exceptions. |
||||
- Validate `setjmp/longjmp`. |
||||
- Validate nesting and repeated invocation of both. |
||||
- Validate support across major browsers. |
||||
|
||||
Exit condition: exception and longjmp behavior is stable, with no unacceptable browser gaps. |
||||
|
||||
### Phase 1: Minimal PHP NTS |
||||
|
||||
Goal: PHP core completes the module and request lifecycle in the browser. |
||||
|
||||
- Cross-compile a minimal php-src. |
||||
- Implement a minimal WASM SAPI or embed validation layer. |
||||
- Support stdout/stderr and MEMFS. |
||||
- Execute a fixed entry point. |
||||
- Validate fatal errors, exceptions, and request shutdown. |
||||
|
||||
Exit condition: executing "success, failure, success" requests in sequence without crashes and without sustained memory growth. |
||||
|
||||
### Phase 2: Integrate PHPX and TypePHP |
||||
|
||||
Goal: The existing TypePHP C++ backend can be compiled by `em++` and statically linked. |
||||
|
||||
- Add a WASM platform/backend configuration to the compiler. |
||||
- Unify compilation flags across PHPX, TypePHP, and third-party libraries. |
||||
- Link a minimal TypePHP `main()`. |
||||
- Establish a WASM smoke PHPT subset. |
||||
- Add capability diagnostics for unsupported system APIs. |
||||
|
||||
Exit condition: basic type, function, class, exception, array, and object tests pass. |
||||
|
||||
### Phase 3: High Precision and Local Streams |
||||
|
||||
Goal: Support TypePHP's critical runtime capabilities. |
||||
|
||||
- Statically link GMP, MPFR, and mpdecimal. |
||||
- Run the full high-precision operator and boundary tests. |
||||
- Support the necessary `file://` and `php://` streams. |
||||
- Add a preloaded read-only resource mechanism. |
||||
- Clearly define all disabled wrappers, transports, and extensions. |
||||
|
||||
Exit condition: high-precision tests pass, local file behavior is deterministic, and all network APIs fail predictably. |
||||
|
||||
### Phase 4: Size, Performance, and Release |
||||
|
||||
Goal: Form a distributable TypePHP WASM SDK. |
||||
|
||||
- Release optimization and dead-code elimination. |
||||
- Review the exported symbol whitelist. |
||||
- Compare allocator and memory growth configurations. |
||||
- Establish download size, startup time, and peak memory benchmarks. |
||||
- Generate `typephp.wasm` and a thin `.mjs` loader. |
||||
- Write user-facing feature and limitation documentation. |
||||
|
||||
Exit condition: artifacts are reproducible, the compatibility checklist is complete, and performance reaches the preset baseline. |
||||
|
||||
### Phase 5: Optional Host Capabilities |
||||
|
||||
These are selected later based on real needs and are not default capabilities of the base runtime: |
||||
|
||||
- IDBFS or OPFS persistence; |
||||
- Explicit HTTP host API; |
||||
- Node.js host; |
||||
- WASI prototype; |
||||
- Multi-instance isolation; |
||||
- Web Worker parallel instances. |
||||
|
||||
Each capability must be enabled through an explicit capability, and PHP code must not obtain all host permissions by default. |
||||
|
||||
## 13. Testing Strategy |
||||
|
||||
### 13.1 Testing Layers |
||||
|
||||
1. **Toolchain tests**: exceptions, longjmp, static libraries, linking, and exported symbols. |
||||
2. **PHP lifecycle tests**: module/request startup, shutdown, bailout, and repeated requests. |
||||
3. **PHPX tests**: Variant, Object, Array, references, exceptions, and resource destruction. |
||||
4. **TypePHP PHPT**: select existing tests that do not depend on the OS, and maintain WASM skip reasons. |
||||
5. **High-precision tests**: full operators, boundaries, errors, and memory stress. |
||||
6. **Capability restriction tests**: network, processes, threads, and dynamic extensions must be stably rejected. |
||||
7. **Browser tests**: minimum supported versions for Chrome, Firefox, and Safari. |
||||
|
||||
### 13.2 Key Metrics |
||||
|
||||
- `.wasm` raw size and compressed size; |
||||
- First instantiation time; |
||||
- Module startup and request startup time; |
||||
- Execution time of a simple TypePHP program; |
||||
- Initial, peak, and post-multiple-request linear memory; |
||||
- Recoverability after exceptions and bailout; |
||||
- JavaScript loader size; |
||||
- Reproducible build checksums for identical inputs. |
||||
|
||||
## 14. Go/No-Go Conditions |
||||
|
||||
If any of the following occurs, pause the full port and re-evaluate the architecture: |
||||
|
||||
- A safe boundary between Zend bailout and C++ stack destruction cannot be established; |
||||
- Request failure stably corrupts subsequent requests, and the one-shot instance model is unacceptable; |
||||
- GMP, MPFR, or mpdecimal require a large-scale invasive fork; |
||||
- `.wasm` size or browser peak memory clearly exceeds the acceptable range of the target scenario; |
||||
- Safari, Firefox, and Chrome require mutually incompatible exception ABIs; |
||||
- Assumptions in PHPX that depend on native threads, dynamic linking, or OS resources cannot be isolated. |
||||
|
||||
If the shortest path is not feasible, then evaluate the standalone WASM runtime/backend described in [BACKEND_NEUTRAL_IR.md](BACKEND_NEUTRAL_IR.md); that rewrite should not be started prematurely without prototype data. |
||||
|
||||
## 15. External References |
||||
|
||||
- [PHP Source Repository](https://github.com/php/php-src) |
||||
- [Emscripten: C setjmp/longjmp Support](https://emscripten.org/docs/porting/setjmp-longjmp.html) |
||||
- [Emscripten: C/C++ Portability Notes](https://emscripten.org/docs/porting/guidelines/portability_guidelines.html) |
||||
- [Emscripten: Code and Memory Optimization](https://emscripten.org/docs/optimizing/Optimizing-Code.html) |
||||
- [WordPress Playground: Compiling PHP to WebAssembly](https://developer.wordpress.org/playground/developers/architecture/wasm-php-compiling/) |
||||
- [WordPress Playground Architecture](https://wordpress.github.io/wordpress-playground/developers/architecture/) |
||||
- [seanmorris/php-wasm](https://github.com/seanmorris/php-wasm) |
||||
- [soyuka/php-wasm](https://github.com/soyuka/php-wasm) |
||||
|
||||
These links are used to track upstream behavior and known porting issues. TypePHP's final implementation and compatibility must be verified by its own build, tests, and benchmarks, and must not directly inherit the conclusions of other projects. |
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,228 @@ |
||||
# Building TypePHP WASI Programs |
||||
|
||||
TypePHP uses the stable WASI 0.2 (Preview 2) and the Component Model. The C++ generated by TypePHP, the PHPX core, a trimmed PHP 8.5 NTS, GMP, MPFR, and mpdecimal are statically linked into a single `.wasm` command or library component. WASI 0.1 (Preview 1) is not supported. |
||||
|
||||
## Environment Requirements |
||||
|
||||
- WASI SDK 33 or higher (LLVM/Clang/LLD 22 or higher) |
||||
- PHP 8.4 or higher, used to run the TypePHP compiler |
||||
- Wasmtime 47 or higher, used to run and test artifacts |
||||
- Jco 1 or higher, used for the browser profile; the component profile does not need Jco |
||||
- wit-bindgen-cli 0.60.0, used for library/WasmExport mode; command mode does not need it |
||||
- The `wasm32-wasip2` integrated SDK bound to the current TypePHP version |
||||
|
||||
The WASI SDK's `bin` directory and Wasmtime must be added to the system `PATH`. The compiler does not probe or use conventional installation directories such as `/opt`, and does not accept dedicated tool directory configuration. WASI static libraries and headers are installed uniformly into PHPX's `wasm/wasm32-wasip2/`: |
||||
|
||||
```bash |
||||
export PATH="<wasi-sdk-bin>:<wasmtime-bin>:$PATH" |
||||
``` |
||||
|
||||
TypePHP uses the existing PHPX locating rules: first read `PHPX_HOME`, then the Composer `swoole/phpx` installation location, and finally `vendor/swoole/phpx`. No WASI-specific environment variable is added. |
||||
|
||||
The WASI build checks `wasm32-wasip2-clang`, `wasm32-wasip2-clang++`, `llvm-ar`, `llvm-ranlib`, `llvm-nm`, `wasm-component-ld`, and `wasmtime`, and confirms the target is `wasm32-unknown-wasip2`. The browser profile additionally checks `jco`, and library mode additionally checks a fixed version of `wit-bindgen`. All tools are found only from `PATH`; npm scripts automatically add the project-local `node_modules/.bin` to `PATH`. |
||||
|
||||
## One-command Build |
||||
|
||||
The source file for command mode must provide `main(): void`: |
||||
|
||||
```php |
||||
<?php |
||||
function main(): void |
||||
{ |
||||
echo "Hello from TypePHP/WASI\n"; |
||||
} |
||||
``` |
||||
|
||||
Run: |
||||
|
||||
```bash |
||||
php bin/tpc.php --wasm hello.php |
||||
``` |
||||
|
||||
Single-file input by default generates only a `hello.wasm` Component in the current directory that can be executed by Wasmtime, without requiring Jco. The generated `.cc` uses the same build directory rules as host mode, defaulting to the TypePHP root's `build/`; it can be overridden with `--build-dir <directory>`. |
||||
|
||||
A project can directly use `project.yml`: |
||||
|
||||
```yaml |
||||
name: wasm-hello |
||||
mode: bin |
||||
wasm: component |
||||
build-dir: build |
||||
output: component/wasm-hello.wasm |
||||
sources: |
||||
- src |
||||
``` |
||||
|
||||
`wasm` only accepts `component` or `browser`, not a boolean value. After configuring, directly run `php bin/tpc.php project.yml` to enter the WASI build without repeatedly passing `--wasm`. WASM projects default to `wasm32-wasip2` when `target-platform` is not configured; `build-dir`, `output`, and `wasm-browser-dir` are all resolved relative to the project file. See `examples/wasm-hello/` for a complete browser application, which explicitly uses `wasm: browser`. |
||||
|
||||
When you need to generate a browser module, configure `wasm: browser` and `wasm-browser-dir`, and ensure Jco is in `PATH`. |
||||
|
||||
The command line can also explicitly select the artifact: |
||||
|
||||
- `--wasm` or `--wasm=component`: generate only a Component runnable by Wasmtime, without checking Jco. |
||||
- `--wasm=browser`: generate the Component and the Jco browser module, requiring `jco` to be in `PATH`. |
||||
|
||||
Detailed configuration such as paths and sources continues to live in `project.yml`, not passed through `--wasm=`. |
||||
|
||||
PHP, PHPX, the TypePHP runtime, GMP, MPFR, and mpdecimal are precompiled into WASI static libraries during the SDK release stage. The application build only compiles the C++ generated by TypePHP for the current program, then links these `.a`. `tpc --wasm` does not download source code, nor does it invoke the build scripts of PHP, PHPX, or high-precision libraries. Library mode invokes `wit-bindgen-cli 0.60.0` from `PATH` to generate the current application's Canonical ABI bindings. |
||||
|
||||
PHP/WASI currently statically includes the `date`, `pcre`, `hash`, `json`, `lexbor`, `random`, `Reflection`, `SPL`, `standard`, `uri`, `ctype`, `calendar`, `bcmath`, `filter`, `tokenizer`, `mbstring`, `zlib`, `fileinfo`, `sodium`, `openssl`, `libxml`, `dom`, `SimpleXML`, `xml`, `xmlreader`, `xmlwriter`, `PDO`, `pdo_sqlite`, `zip`, `bz2`, and `exif` extensions. OpenSSL uses a crypto-only build without the TLS stream transport; HTTP/HTTPS is still provided by the WASI HTTP Component. |
||||
|
||||
Every C/C++ translation unit uniformly uses standard Wasm C++ exceptions and WASI SJLJ; the linking stage treats ABI warnings as errors, and old 32-bit `zend_long` caches are automatically invalidated. |
||||
|
||||
Run: |
||||
|
||||
```bash |
||||
wasmtime hello.wasm |
||||
``` |
||||
|
||||
Chrome Demo: |
||||
|
||||
```bash |
||||
cd examples/wasm-hello |
||||
npm ci |
||||
npm run wasm |
||||
npm run dev |
||||
``` |
||||
|
||||
The browser side always executes the Component in a dedicated Worker. The in-memory filesystem is used by default; after the startup message sent to the Worker sets `persistent: true`, the filesystem snapshot is restored and saved through OPFS at startup and exit. During program execution the synchronous in-memory filesystem is still used, avoiding crossing the asynchronous JS boundary on every PHP file access. |
||||
|
||||
## ZendVM Lifecycle for Command and Library |
||||
|
||||
### Command Mode |
||||
|
||||
Command mode has a generated C++ `main()` entry. The entry calls in sequence: |
||||
|
||||
```text |
||||
typephp_<project>_runtime_init(argc, argv) |
||||
→ php_embed_init() |
||||
→ PHP/SAPI module startup and MINIT |
||||
→ PHP request startup and RINIT |
||||
→ register and start the current TypePHP application module |
||||
→ MINIT and RINIT of the current application |
||||
|
||||
execute TypePHP main() |
||||
|
||||
typephp_<project>_runtime_shutdown() |
||||
→ RSHUTDOWN and module cleanup of the current application |
||||
→ php_embed_shutdown() |
||||
→ PHP request/module/SAPI shutdown |
||||
``` |
||||
|
||||
The caller does not need to be aware of these steps, because the generated native `main()` automatically wraps the entire program lifecycle. |
||||
|
||||
### Library Mode Must Create the Runtime Resource First |
||||
|
||||
A library component has no `main()` to execute automatically; merely instantiating the `.wasm` only completes the instantiation of the Component and the C/C++ Runtime, and does not mean the ZendVM request is already available. The Host must first call the generated WIT function: |
||||
|
||||
```wit |
||||
create-runtime: func() -> result<runtime, typephp-error>; |
||||
``` |
||||
|
||||
The corresponding call in the browser is: |
||||
|
||||
```js |
||||
const component = await instantiate(null, wasi.getImportObject()); |
||||
const runtime = await component.api.createRuntime(); |
||||
|
||||
try { |
||||
const result = await runtime.someExportedFunction(); |
||||
} finally { |
||||
runtime[Symbol.dispose](); |
||||
} |
||||
``` |
||||
|
||||
Internally, `createRuntime()` calls the project-level initialization symbol through `TYPEPHP_RUNTIME_INIT(<project>)(1, argv)`. The Host |
||||
only needs to call this one stable interface, and should not directly call `php_embed_init()`, MINIT, RINIT, or any Zend C API. |
||||
|
||||
The current initialization order is as follows: |
||||
|
||||
1. `php_embed_init()` initializes the Embed SAPI, the PHP core, and static extensions, and starts a PHP request; the PHP core and the already-registered static extensions complete MINIT/RINIT here. |
||||
2. Set up PHPX's exception bridge so that PHP exceptions can safely return into the generated WIT `result`. |
||||
3. Obtain the current TypePHP application's `zend_module_entry`, call `zend_register_module_ex()` and `zend_startup_module_ex()` to complete application module registration and MINIT. |
||||
4. Register standard streams and set SAPI request information such as the request path. |
||||
5. Because the Embed request and request memory pool are already started at this point, the generated code explicitly calls the current application module's `request_startup_func` to perform that module's RINIT; RINIT then initializes TypePHP request-level globals and class static data, and only returns the `runtime` resource after completion. |
||||
|
||||
What is "manually" called here is the Host-visible `create-runtime()`, not letting the user manually assemble the ZendVM lifecycle. The specific calls of MINIT/RINIT and their order are all encapsulated in PHPX and the generated Component adapter. |
||||
|
||||
### Exported Calls Share the Same Request |
||||
|
||||
All `#[WasmExport]` calls on the same `runtime` resource share the Zend request established by a single RINIT: |
||||
|
||||
- RINIT/RSHUTDOWN are not executed repeatedly before and after each function call. |
||||
- The PHP request memory pool, request-level globals, and static state persist until the resource is released. |
||||
- Currently only NTS is supported; calls on the same runtime must be serial, and the generated adapter rejects concurrent or re-entrant calls. |
||||
- Ordinary PHP exceptions are converted into WIT `result` errors, and the runtime can still continue to be used. |
||||
- A Zend bailout indicates that request state may already be corrupted, so the adapter marks the runtime as failed, and subsequent calls are rejected until the resource is released. |
||||
|
||||
### RSHUTDOWN Is Only Executed When the Resource Is Released |
||||
|
||||
Releasing the WIT `runtime` resource calls the project-level shutdown symbol through `TYPEPHP_RUNTIME_SHUTDOWN(<project>)()`: |
||||
|
||||
1. Call the current TypePHP application module's RSHUTDOWN to clean up TypePHP request-level objects and global data. |
||||
2. Unregister and shut down the current application module, performing the corresponding module cleanup. |
||||
3. Call `php_embed_shutdown()` to complete request shutdown, module shutdown, and SAPI shutdown of the remaining extensions. |
||||
4. Finally release the request memory pool to avoid PHP/CPP wrapper objects continuing to destruct after the memory pool disappears. |
||||
|
||||
Do not rely only on JavaScript GC to trigger the resource finalizer. Browser and Node Hosts should explicitly call `runtime[Symbol.dispose]()` in a `finally` block; Wasmtime or other Host bindings should also explicitly drop the resource. Directly terminating a Worker or process reclaims the entire Wasm instance, but does not guarantee that PHP's RSHUTDOWN/MSHUTDOWN callbacks are executed, so data that must be persisted should not be placed only in shutdown callbacks. |
||||
|
||||
A Component instance currently allows only one active runtime resource at a time. After release it can be recreated; when initialization fails or a Zend bailout occurs, the old resource should be released first, rather than continuing to call exported functions. |
||||
|
||||
## High-precision Types |
||||
|
||||
WASI artifacts include TypePHP's three language-level high-precision types: |
||||
|
||||
- `BigInt`: GMP 6.3.0 |
||||
- `BigFloat`: MPFR 4.2.2 |
||||
- `Decimal`: mpdecimal 4.0.1 |
||||
|
||||
A complete example is at [high-precision.php](../examples/high-precision.php). Build and run: |
||||
|
||||
```bash |
||||
php bin/tpc.php --wasm examples/high-precision.php |
||||
wasmtime -S http high-precision.wasm |
||||
``` |
||||
|
||||
Expected output: |
||||
|
||||
```text |
||||
1111111101111111110111111111010 |
||||
1000000000000000000000000000001 |
||||
12348.14159265358979324 |
||||
``` |
||||
|
||||
wasm32 uses 32-bit pointers, but PHP's `zend_long` remains 64-bit to maintain TypePHP's integer semantics with 64-bit PHP. GMP and mpdecimal use 32-bit limbs; this does not change arbitrary-precision semantics, but large-number throughput is lower than a native 64-bit build with assembly optimization. |
||||
|
||||
## Current Platform Boundaries |
||||
|
||||
- Only NTS, single-threaded is supported. |
||||
- Fiber and Generator are disabled; the compiler reports a fatal error directly when it finds `yield`. |
||||
- The PHPX Facade API is entirely disabled under `__wasi__`. PHPX core types and `phpx_std` are still usable. |
||||
- Dynamic extensions, network sockets, processes, shells, and signals are not supported. Statically recognizable calls report a fatal error at compile time. |
||||
- The PHP stream framework, local file capability, and time and random number capabilities provided by the WASI host are retained. |
||||
- A command component can be run directly by Wasmtime; a library component requires the Host to call `create-runtime()` and exported functions according to the WIT interface. Chrome uses the ESM generated by Jco and the Worker host in `examples/wasm-hello/typephp-worker.mjs`. |
||||
|
||||
The PHPX Facade is only a convenience wrapper generated for PHP optional extensions, not part of the TypePHP ABI. Disabling it entirely under WASI avoids exposing unavailable APIs such as curl, socket, and Swoole as interfaces that "compile but fail to link"; the PHP/WASI statically built-in extensions themselves are not affected by the Facade switch. |
||||
|
||||
## WASI SDK Directory |
||||
|
||||
The integrated SDK uses a single, complete prefix located at `wasm/wasm32-wasip2/` in the PHPX root directory: |
||||
|
||||
```text |
||||
phpx/wasm/wasm32-wasip2/ |
||||
├── include/php/ # PHP installed headers |
||||
├── include/phpx/ # PHPX and TypePHP runtime headers |
||||
├── include/gmp.h ... |
||||
├── lib/libphp.a |
||||
├── lib/libphpx.a |
||||
├── lib/libgmp.a |
||||
├── lib/libgmpxx.a |
||||
├── lib/libmpfr.a |
||||
├── lib/libmpdec.a |
||||
├── lib/libmpdec++.a |
||||
└── .typephp-wasi-sdk-abi |
||||
``` |
||||
|
||||
Ordinary users obtain this directory through the TypePHP/PHPX integrated installation package. TypePHP developers need to clone the `php-8.5.9-wasm` and PHPX source bound to the current version, and assemble the full SDK through `wasm/build-sdk.sh`. PHP/WASI is only responsible for PHP; PHPX is responsible for GMP, MPFR, its dedicated mpdecimal, and the PHPX runtime. All artifacts are installed into the same PHPX checkout. If PHPX is not at `vendor/swoole/phpx`, continue using the existing `PHPX_HOME` to point to that checkout. |
||||
|
||||
No path to separately override `libphp.a`, `libphpx.a`, or the numeric libraries is provided; all libraries, headers, and `.typephp-wasi-sdk-abi` must come from the same compatible build, avoiding mixing different `zend_long`, C++ exceptions, SJLJ, or Component Model ABIs. |
||||
@ -0,0 +1,143 @@ |
||||
# yield / Generator |
||||
|
||||
TypePHP compiles functions, methods, anonymous functions, and arrow functions that contain `yield` or `yield from` into Fiber generators. Calling them does not immediately execute the generator function body; instead it returns a global `\FiberGenerator` object. The Fiber is created and started only on the first iteration, or when `current()`, `valid()`, `send()`, or `throw()` is called. |
||||
|
||||
The generated C++ function body runs inside the Fiber. Each `yield` hands its key/value to the iteration driver via `Fiber::suspend()`, and resumes the original C++ call stack on `next()`, `send()`, or `throw()`. The runtime distinguishes not-started, running, suspended, normally-returned, and exceptionally-closed states via `NEW`, `RUNNING`, `SUSPENDED`, `CLOSED_RETURNED`, and `CLOSED_FAILED`. |
||||
|
||||
## Iteration interop |
||||
|
||||
The following combinations already work: |
||||
|
||||
| Driver | Iterated object | |
||||
|---|---| |
||||
| TypePHP Native `foreach` | TypePHP Native generator | |
||||
| TypePHP Native `foreach` | Zend `Generator` returned by dynamic PHP | |
||||
| ZendVM `foreach` | `\FiberGenerator` returned by a TypePHP Native generator | |
||||
| TypePHP `yield from` | arrays, `Iterator`, `IteratorAggregate`, Zend `Generator`, `\FiberGenerator` | |
||||
|
||||
`\FiberGenerator` implements `Iterator` with an interface signature consistent with PHP: |
||||
|
||||
```php |
||||
rewind(): void |
||||
next(): void |
||||
valid(): bool |
||||
current(): mixed |
||||
key(): mixed |
||||
send(mixed $value): mixed |
||||
throw(Throwable $exception): mixed |
||||
getReturn(): mixed |
||||
``` |
||||
|
||||
The current state machine already covers first and repeated `rewind()`, `next()`/`send()`/`throw()` when not started, calls after closure, automatic integer keys, normal and exceptional `getReturn()`, exception propagation, and executing `finally` when destructing in a suspended state. |
||||
|
||||
## Differences from PHP |
||||
|
||||
### Not a Zend Generator |
||||
|
||||
TypePHP generators return the global `\FiberGenerator`, not PHP's built-in `Generator`: |
||||
|
||||
```php |
||||
$generator instanceof Iterator; // true |
||||
$generator instanceof Generator; // false |
||||
``` |
||||
|
||||
Therefore the following differences exist: |
||||
|
||||
- A generator function cannot declare the precise return type `Generator`. |
||||
- `Iterator`, `Traversable`, `iterable`, `object`, `mixed`, or union types containing these compatible types can be used. |
||||
- `ReflectionGenerator` only accepts Zend `Generator` and cannot be used with `\FiberGenerator`. |
||||
- `get_class()`, Reflection class info, and the class name in exception stacks differ from Zend `Generator`. |
||||
- No guarantee that `var_dump()`, debug properties, or the internal object layout match Zend `Generator`. |
||||
- `\FiberGenerator` is a final runtime-internal type; business code is forbidden from directly instantiating, inheriting, cloning, or serializing it. |
||||
|
||||
### By-reference Generators not supported |
||||
|
||||
The following PHP syntax is not yet supported: |
||||
|
||||
```php |
||||
function &values(): iterable |
||||
{ |
||||
yield $value; |
||||
} |
||||
|
||||
foreach (values() as &$value) { |
||||
} |
||||
``` |
||||
|
||||
TypePHP does not support: |
||||
|
||||
- generator functions or methods returning by reference. |
||||
- by-reference yield semantics. |
||||
- by-reference `foreach` over a generator. |
||||
- maintaining element reference identity through a generator. |
||||
|
||||
`current()`, `send()`, `throw()`, and `getReturn()` all return ordinary PHP values; the runtime unwraps `INDIRECT` and `REFERENCE` wrappers and does not return reference containers. |
||||
|
||||
### Parameter limitations |
||||
|
||||
TypePHP generators do not yet support the following parameter declarations: |
||||
|
||||
- by-reference parameters, e.g. `function values(&$value)`. |
||||
- variadic parameters, e.g. `function values(...$values)`. |
||||
- by-reference variadic parameters, e.g. `function values(&...$values)`. |
||||
|
||||
Ordinary parameters, defaults, union-typed parameters, object parameters, and `$this` in methods can be used. Parameter type checks and constructor property promotion execute when the generator object is created, while the function body remains lazily executed. |
||||
|
||||
### Traversable boundary |
||||
|
||||
`yield from` and TypePHP Native object `foreach` use different underlying paths. `foreach` uniformly drives arrays, ordinary objects, and Zend `Traversable` through PHPX `ForeachIterator`; `yield from` still performs delegation within the generator itself. |
||||
|
||||
This means: |
||||
|
||||
- userland `Iterator`, `IteratorAggregate`, Zend `Generator`, and internal `Traversable` provided by extensions are all iterated through the class's `get_iterator` handler. |
||||
- ordinary objects directly traverse the live property table, performing public, protected, and private visibility checks under the current TypePHP class scope. |
||||
- `foreach ($iterable as $value)` that does not read the key does not call `Iterator::key()`. |
||||
- TypePHP `yield from` detects `IteratorAggregate::getIterator()` returning itself or forming an object cycle, and throws an exception. |
||||
- TypePHP Native `foreach` delegates `IteratorAggregate` unwrapping and cycle detection to the Zend iterator handler, keeping behavior consistent with the current PHP runtime. |
||||
- the exception type, message text, and stack info for an invalid `getIterator()` return value may not fully match ZendVM. |
||||
|
||||
### Fiber observable differences |
||||
|
||||
Zend `Generator` is a dedicated ZendVM execution object; TypePHP generators use PHP Fibers to save the full C/C++ stack, so: |
||||
|
||||
- the runtime environment must provide PHP Fibers. |
||||
- `Fiber`, internal closures, or TypePHP runtime frames may appear in exception stacks. |
||||
- file names, line numbers, and call-stack shapes are not guaranteed to exactly match `ReflectionGenerator` or Zend Generator. |
||||
- normal iteration, exception propagation, and `finally` during suspended destructors already have regression tests, but execution order for complex object cycles, request shutdown, process exit, and destructors that throw again may still differ from Zend Generator. |
||||
- when a Fiber is forcibly closed, Zend's internal graceful-exit is used to unwind the C++ stack; this object is not a public exception type that business code can catch or rely on. |
||||
|
||||
### yield from differences |
||||
|
||||
Key/value forwarding for arrays, ordinary Iterators, and generators, generator return values, and `send()`/`throw()` delegation are implemented, but the underlying implementation is not the Zend `yield from` opcode: |
||||
|
||||
- delegation is done through the `rewind()`, `valid()`, `key()`, `current()`, `next()`, `send()`, `throw()`, and `getReturn()` methods. |
||||
- side effects, exception stacks, and call counts produced by custom Iterator methods should avoid depending on Zend Generator's internal implementation details. |
||||
- `yield from` over a non-generator Iterator yields `null`; only Zend `Generator` and `\FiberGenerator` read `getReturn()`. |
||||
|
||||
## Performance differences |
||||
|
||||
Fiber generators do not change the generated code for ordinary array or ordinary container `foreach`. Extra cost arises only when a generator is actually created and driven. |
||||
|
||||
Each yield currently requires: |
||||
|
||||
- Fiber suspend/resume. |
||||
- `Iterator` method calls. |
||||
- generator state and object property reads/writes. |
||||
- creation and release of the key/value payload array. |
||||
- additional delegation calls in the `yield from` scenario. |
||||
|
||||
Therefore TypePHP Fiber generators are usually slower than Native C++ array `foreach`, and may also be slower than Zend's dedicated Generator opcode. High-frequency, short-element iteration should prefer arrays or Native containers; generators are more suited to lazy computation, streaming, and scenarios that need to preserve the full Native call stack. |
||||
|
||||
## Incompatibility checklist |
||||
|
||||
The following PHP behaviors currently cannot be relied upon: |
||||
|
||||
- the returned object being a Zend `Generator`. |
||||
- `instanceof Generator`. |
||||
- declaring the precise `Generator` return type. |
||||
- `ReflectionGenerator`. |
||||
- by-reference generator returns, by-reference yield, or by-reference foreach. |
||||
- by-reference or variadic generator parameters. |
||||
- debug output and internal properties identical to Zend Generator. |
||||
- all internal extension `Traversable` being iterable through Zend iterator handlers. |
||||
- Fiber closure, complex destruction, and process exit with exactly the same stack and destruction order as Zend Generator. |
||||
@ -0,0 +1,432 @@ |
||||
# AOT Compiler Optimization Priority Re-evaluation: Considering GCC/Clang Secondary Compilation |
||||
|
||||
## Core Principle |
||||
|
||||
The optimization value of AOT does not lie in doing what GCC/Clang already do, but in **providing information that GCC/Clang cannot infer from the C++ code**. |
||||
|
||||
What GCC/Clang can already do under `-O2`/`-O3`: |
||||
- Constant folding, constant propagation (SCCP) |
||||
- Dead code elimination (DCE) |
||||
- Common subexpression elimination (CSE) |
||||
- Loop unrolling, vectorization |
||||
- Instruction selection, register allocation |
||||
- Function inlining (within the same translation unit) |
||||
- Branch prediction optimization |
||||
- SIMD auto-vectorization |
||||
|
||||
What GCC/Clang **cannot do** — because the semantics are obscured by abstraction layers such as `php::Var`, `php::Object`, and virtual function calls: |
||||
- Narrow `php::Var` to a concrete C++ type |
||||
- Eliminate virtual calls of `php::Object` |
||||
- Eliminate reference counting operations |
||||
- Move `php::Array` from heap allocation to stack allocation |
||||
- Downgrade `php::BigInt` to native `int64_t` |
||||
- Eliminate the function lookup/dispatch overhead of `php::call` |
||||
- Cross-translation-unit global analysis (LTO can partially do this, but is limited by visibility) |
||||
|
||||
--- |
||||
|
||||
## Reordered Priorities |
||||
|
||||
### Tier 1: Directly Improve C++ Code Shape (GCC Cannot Fix) |
||||
|
||||
These optimizations change **what C++ code is generated**, not the C++ code that has already been optimized. This is the core value of AOT. |
||||
|
||||
#### #1 Type Inference → Generate Concrete C++ Types Instead of php::Var |
||||
|
||||
**Problem:** Currently AOT maps a large number of variables to `php::Var` (a general type), and GCC can only perform limited optimization on the member functions of `php::Var`. |
||||
|
||||
**Benefit:** |
||||
``` |
||||
// Currently generated code (GCC can't do much) |
||||
php::Var a = php::toInt(x); |
||||
php::Var b = php::toInt(y); |
||||
php::Var c = php::add(a, b); // GCC can't see this is addition |
||||
|
||||
// After type inference |
||||
int64_t a = php::toInt(x); |
||||
int64_t b = php::toInt(y); |
||||
int64_t c = a + b; // GCC can perform all integer optimizations |
||||
``` |
||||
|
||||
**Value amplification for GCC secondary compilation:** once the type becomes `int64_t`, GCC can further: |
||||
- Register allocation (no longer through `php::Var`'s memory layout) |
||||
- Constant folding and propagation |
||||
- Loop optimization (operations on `php::Var` inside loops become pure integer arithmetic) |
||||
- Automatic SIMD |
||||
|
||||
**Implementation recommendation:** this is the **highest priority**. A complete SSA form is not needed; just infer the most precise type at each assignment point and use that type to declare the variable during C++ code generation. The existing union/nullable type check infrastructure can be extended into type inference. |
||||
|
||||
--- |
||||
|
||||
#### #2 Devirtualization |
||||
|
||||
**Problem:** all non-final method calls in PHP are virtual calls. AOT generates `obj->method()` (a C++ virtual function call) even if only one subclass implements the method. |
||||
|
||||
**Benefit:** |
||||
``` |
||||
// Currently generated (virtual call, GCC dares not eliminate the vtable lookup) |
||||
return self->foo(); // virtual call |
||||
|
||||
// After devirtualization (GCC can inline this call!) |
||||
return Aot_MyClass_foo(self); // direct function call |
||||
``` |
||||
|
||||
**Breakthrough:** collect all classes and their inheritance relationships at compile time. If: |
||||
- The method is private → always call directly |
||||
- The method is final → always call directly |
||||
- The class has only 1 non-abstract implementation across all compiled files → call directly |
||||
- `self::foo()` calls a method of its own class → can call directly (if the class has no undiscovered subclasses) |
||||
|
||||
**Synergy with GCC:** once it becomes a direct call, GCC can: |
||||
- Inline the entire function body |
||||
- Cross-function constant propagation |
||||
- Chain-eliminate subsequent redundant operations |
||||
|
||||
--- |
||||
|
||||
#### #3 Escape Analysis → Stack Allocation + Reference Count Elimination |
||||
|
||||
**Problem:** every `new` object/array is allocated on the heap and its lifetime is managed through reference counting. |
||||
|
||||
**Benefit:** |
||||
``` |
||||
// Currently (heap allocation + refcount) |
||||
php::Array arr = php::newArray(); |
||||
arr.set("key", value); // refcount operations |
||||
return arr; // copy + refcount |
||||
|
||||
// After escape analysis (stack allocation + no refcount) |
||||
zend_array arr; // on the stack |
||||
zend_hash_update(&arr, "key", value); // no refcount |
||||
return php::Array::fromStack(std::move(arr)); // pack only at the return point |
||||
``` |
||||
|
||||
**Impact on GCC secondary compilation:** this is the optimization that helps GCC the **most**, because: |
||||
- Heap allocation → stack allocation: eliminates `malloc` calls, and GCC can fully optimize the stack layout |
||||
- Eliminating refcount: GCC does not need to analyze the side effects of interlocked operations and can freely reorder instructions |
||||
- GCC can perform SROA (Scalar Replacement of Aggregates) on stack variables, breaking arrays/objects into scalars |
||||
|
||||
**Implementation recommendation:** even simple local escape analysis (only analyzing whether an object is passed outside the function) can eliminate a large number of allocations. The complex version (cross-function escape analysis) continues to increase the benefit. |
||||
|
||||
--- |
||||
|
||||
#### #4 Call-Graph-Driven Cross-File Inlining |
||||
|
||||
**Problem:** the same PHP project may be compiled into multiple `.cc` files; GCC's LTO can inline across files but is limited by compilation time. |
||||
|
||||
**Benefit:** AOT knows the entire call graph at compile time and can **decide inlining while generating C++ code**: |
||||
|
||||
``` |
||||
// Currently |
||||
auto result = aot_smallHelper(x, y); // function call, even if the body is one line |
||||
|
||||
// After inlining |
||||
auto result = x + y; // GCC can continue optimizing |
||||
``` |
||||
|
||||
**Key decision information:** |
||||
- Function body size (< 10 lines → inline candidate) |
||||
- Call count (called only once → inlining can fully eliminate the function) |
||||
- Recursion marker (recursive functions are not inlined) |
||||
- Cross-function constant propagation opportunities (arguments are constants → after inlining GCC can fold the entire function) |
||||
|
||||
**Synergy with GCC:** AOT makes the "decision" (whether to inline) and generates the inlined code directly at the call site. GCC continues optimizing on the larger inlined body. AOT possesses information that GCC does not (a global view from the call graph). |
||||
|
||||
--- |
||||
|
||||
### Tier 2: Provide More Precise Type Information to GCC |
||||
|
||||
These optimizations improve the quality of the C++ types passed to GCC. |
||||
|
||||
#### #5 Integer Range Inference → Choose the Optimal Integer Type |
||||
|
||||
**Benefit:** |
||||
```php |
||||
// PHP source: loop counter, 0 to 100 |
||||
for ($i = 0; $i < 100; $i++) { ... } |
||||
|
||||
// Currently generated by AOT |
||||
int64_t i = 0; // always uses int64_t (to prevent overflow) |
||||
|
||||
// After range inference |
||||
int8_t i = 0; // 0..100 is enough, better cache locality |
||||
// or at least |
||||
int64_t i = 0; // but marked "no overflow possible", no BigInt promotion inserted |
||||
``` |
||||
|
||||
**Impact on GCC:** |
||||
- Smaller types → better vectorization (more elements packed into SIMD registers) |
||||
- "Will not overflow" assertions → GCC's VRP (Value Range Propagation) can perform more aggressive optimizations based on this premise |
||||
|
||||
#### #6 Avoid Unnecessary BigInt/BigFloat Allocation |
||||
|
||||
**Related to #5.** PHP's integer arithmetic automatically promotes to float or BigInt on overflow. If AOT can prove no overflow, no promotion code needs to be generated. |
||||
|
||||
``` |
||||
// Currently (every int operation must consider overflow) |
||||
php::Var result = php::BigInt::add(php::toBigInt(a), php::toBigInt(b)); |
||||
|
||||
// After range inference (no overflow) |
||||
int64_t result = a + b; // pure integer, GCC's world |
||||
``` |
||||
|
||||
--- |
||||
|
||||
### Tier 3: Partially Overlapping with GCC but Still Valuable |
||||
|
||||
#### #7 Constant Folding (PHP Level) |
||||
|
||||
**What GCC can do:** constant expressions at the C++ level are fully folded in GCC's SCCP pass. |
||||
|
||||
**AOT's unique value:** |
||||
- PHP-specific constants: `PHP_INT_MAX`, `PHP_VERSION`, `__DIR__`, etc. are fully resolved at compile time |
||||
- Constant resolution across PHP namespaces/class names (GCC cannot see PHP's symbol semantics) |
||||
- Results of PHP built-in functions: `strlen("hello")` → 5 (GCC does not know `strlen`'s internal implementation but can fold functions with known arguments) |
||||
|
||||
**Assessment:** limited value. Most PHP source code does not contain complex compile-time-evaluable constant expressions. |
||||
|
||||
#### #8 Dead Code Elimination (PHP Level) |
||||
|
||||
**What GCC can do:** GCC's DCE + unreachable block elimination is very mature. |
||||
|
||||
**AOT's unique value:** |
||||
- Eliminate branches based on the PHP type system: `if (false)` eliminated at the PHP level → no C++ code generated |
||||
- Eliminate branches based on type narrowing: `if ($x instanceof Foo)` when `$x`'s type is already determined to be `Bar` and `Foo ⊄ Bar`, the false branch is unreachable |
||||
- Eliminate uncalled PHP functions/classes (cross-file dead code) |
||||
|
||||
**Assessment:** medium value. AOT should focus on "DCE based on PHP semantics" rather than competing with GCC on "DCE based on C++ semantics". |
||||
|
||||
--- |
||||
|
||||
### Tier 4: Improve the Compilation Process Rather than Output Quality |
||||
|
||||
#### #9 File Caching (Compilation Acceleration) |
||||
|
||||
**What GCC can do:** ccache. But ccache requires file content hash matching. |
||||
|
||||
**AOT's unique value:** |
||||
- Cache PHP→C++ translation results (parsed AST + type information) |
||||
- Reuse across restarts |
||||
- Incremental updates (only recompile changed PHP files) |
||||
|
||||
**Assessment:** useful but not core. First optimize the output code quality, then optimize compilation speed. |
||||
|
||||
#### #10 Pass Pipeline Architecture |
||||
|
||||
Infrastructure at the engineering architecture level, improving code maintainability and extensibility. Does not directly affect output quality. |
||||
|
||||
--- |
||||
|
||||
## Priority Summary |
||||
|
||||
``` |
||||
Must do (directly affects the C++ types GCC sees): |
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ |
||||
⭐ #1 Type inference → avoid php::Var generalization 【largest single-point benefit】 |
||||
⭐ #2 Devirtualization → eliminate virtual call overhead 【second-largest benefit】 |
||||
⭐ #3 Escape analysis → stack allocation + eliminate refcount 【great benefit for numeric code】 |
||||
|
||||
Strongly recommended (give GCC better premises): |
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ |
||||
⭐ #4 Call-graph inlining → cross translation-unit boundaries |
||||
⭐ #5 Range inference → optimal integer type selection |
||||
⭐ #6 Eliminate unnecessary BigInt/BigFloat allocation |
||||
|
||||
Nice to have (unique value but not core): |
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ |
||||
#7 PHP-level constant folding |
||||
#8 PHP-semantic-level DCE |
||||
#9 Compilation cache acceleration |
||||
|
||||
Engineering foundation: |
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ |
||||
#10 Pass Pipeline architecture |
||||
``` |
||||
|
||||
## Further Analysis of #1 Type Inference |
||||
|
||||
This is the largest single-point benefit item and deserves further elaboration. The following is the concrete technical path: |
||||
|
||||
### Key Insight: No Complete SSA Needed |
||||
|
||||
AOT **generates C++ code**, not optimized bytecode. This means: |
||||
|
||||
1. No φ functions needed: C++ variable assignment is naturally the "fresh variable" form of SSA |
||||
2. No dominance frontier computation needed: the C++ compiler does it itself |
||||
3. Only need to compute the possible type of **each expression** and generate the corresponding C++ type declaration |
||||
|
||||
### Current Problem |
||||
|
||||
``` |
||||
PHP source: $a = 1; |
||||
$b = $a + 2; |
||||
Generated C++: php::Var a = php::toInt(php::toVar(1)); |
||||
php::Var b = php::add(a, php::toVar(2)); // !!all Var!! |
||||
``` |
||||
|
||||
What GCC sees at this point: |
||||
- All values are `php::Var` (a complex struct) |
||||
- Operations go through the `php::add()` function (external symbol, not inlinable) |
||||
- Cannot determine types, cannot narrow |
||||
|
||||
### Target |
||||
|
||||
``` |
||||
PHP source: $a = 1; |
||||
$b = $a + 2; |
||||
Generated C++: int64_t a = 1; |
||||
int64_t b = a + 2; // GCC can fully optimize |
||||
``` |
||||
|
||||
### Implementation Path |
||||
|
||||
**Phase A: Local type propagation (~300 lines)** |
||||
|
||||
Traverse the AST, at each assignment point: |
||||
1. If the RHS is a literal → type is clear, record it |
||||
2. If the RHS is a variable of known type → propagate the type |
||||
3. If the RHS is a binary operation and both operand types are known → infer the result type |
||||
4. If a variable is assigned different types multiple times → degrade to `php::Var` |
||||
|
||||
**Phase B: Conditional narrowing (~200 lines, depends on TypeSpecifier)** |
||||
|
||||
``` |
||||
if ($x instanceof MyClass) { // entering this branch, $x's value is determined to be MyClass |
||||
$x->method(); // direct call can be used here |
||||
} |
||||
``` |
||||
|
||||
**Phase C: Cross-basic-block merge (~200 lines)** |
||||
|
||||
``` |
||||
if (cond) { $x = 1; } else { $x = 2; } |
||||
// merge point: $x is int64_t (both branches are integers) |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## Safety Constraints on Type Narrowing: PHP instanceof Semantics Analysis |
||||
|
||||
### Problem |
||||
|
||||
The user pointed out: the semantics of `if ($x instanceof Foo)` in PHP is that `$x` may be an instance of `Foo` **or any of its subclasses**. This is the same as C++'s `dynamic_cast<Foo*>`. |
||||
|
||||
This means: |
||||
|
||||
```php |
||||
if ($x instanceof Foo) { |
||||
$x->method(); // virtual call! $x's actual class may override method() |
||||
} |
||||
``` |
||||
|
||||
You **cannot** turn `$x->method()` into a direct call to `Foo::method()` just because it was narrowed to `Foo` — unless `Foo` is a final class or method is private. |
||||
|
||||
### Three Safety Levels of Narrowing |
||||
|
||||
#### Level 1: Type member accessibility (always safe) |
||||
|
||||
Even without knowing the exact type, as long as you know `$x` is a subtype of `Foo`, you can: |
||||
- Confirm `$x` has access to a certain method/property |
||||
- Eliminate impossible code paths (`if ($x instanceof Foo)` is false when `$x` is known to be `Bar` and `Foo` and `Bar` are unrelated) |
||||
- Generate correct C++ type annotations (`Foo*` instead of `void*`/`php::Var`) |
||||
|
||||
```php |
||||
// Before narrowing: $x is php::Var, calling foo() may throw an exception |
||||
// After narrowing: $x is some subtype of Foo, definitely has a foo() method |
||||
$x->foo(); // can confirm the method exists (though still a virtual call) |
||||
``` |
||||
|
||||
**Benefit:** eliminate runtime checks for nonexistent methods, improve error detection (found at compile time). |
||||
|
||||
#### Level 2: final class + private method (fully safe) |
||||
|
||||
| Scenario | Can narrow to exact type? | Can call directly? | |
||||
|------|-------------------|------------| |
||||
| `$x instanceof FinalClass` | Yes | Yes | |
||||
| `$x->privateMethod()` | Yes (private cannot be overridden) | Yes | |
||||
| `self::method()` and class has no subclasses | Yes | Yes | |
||||
| `$x instanceof NonFinalClass` | No | No | |
||||
|
||||
```php |
||||
final class Logger { |
||||
public function log(string $msg): void { ... } |
||||
} |
||||
|
||||
if ($x instanceof Logger) { |
||||
$x->log("test"); // safe: direct call to Aot_Logger_log($x, "test") |
||||
} |
||||
``` |
||||
|
||||
#### Level 3: Finite subclass set + guard (guard-based devirtualization) |
||||
|
||||
When `Foo` is not final, but AOT compiled the entire codebase and can determine **all subclasses of Foo**: |
||||
|
||||
``` |
||||
Known class hierarchy: |
||||
Foo (abstract) |
||||
├── SubFooA method() implementation |
||||
└── SubFooB method() implementation |
||||
``` |
||||
|
||||
At this point `$x instanceof Foo` means `$x` can only be `SubFooA` or `SubFooB`. It can generate: |
||||
|
||||
```cpp |
||||
// guard-based devirtualization |
||||
if (x.getInstanceOf(get_class(SubFooA))) { |
||||
Aot_SubFooA_method(x); // direct call |
||||
} else { |
||||
Aot_SubFooB_method(x); // direct call (the last one needs no check) |
||||
} |
||||
// replaces the original: x->method(); (virtual call) |
||||
``` |
||||
|
||||
**Benefit:** for class hierarchies with a known finite subclass set, replace vtable lookup with if-else dispatch. If there is only 1 subclass (de facto final), the branch is fully eliminated. |
||||
|
||||
### How PHP Itself Handles This |
||||
|
||||
The `zend_ssa_var_info` structure of Zend SSA precisely tracks this distinction: |
||||
|
||||
```c |
||||
typedef struct _zend_ssa_var_info { |
||||
uint32_t type; |
||||
zend_class_entry *ce; |
||||
bool is_instanceof : 1; // 0 = class == ce, 1 = may be child of ce |
||||
// ... |
||||
} zend_ssa_var_info; |
||||
``` |
||||
|
||||
- `is_instanceof = false`: the variable is **exactly this class** (from a `new Foo()` or a `get_class($x) === 'Foo'` check) |
||||
- `is_instanceof = true`: the variable **may be this class or its subclass** (from `$x instanceof Foo`) |
||||
|
||||
Only when `is_instanceof = false` and ce is non-abstract can it be safely devirtualized. |
||||
|
||||
### Impact on Tier 1 Optimizations |
||||
|
||||
| Optimization | Affected by instanceof semantics? | Valid scope | |
||||
|------|------------------------|---------| |
||||
| #1 Type inference (generate concrete C++ types) | Not directly affected | Narrowing to `Foo*` is still valuable (better than `php::Var`) | |
||||
| #2 Devirtualization | **Yes — instanceof does not provide exact type** | Only final classes, private methods, finite subclass guards | |
||||
| #3 Escape analysis | Not directly affected | Does not need exact type, only escape/non-escape determination | |
||||
| #4 Call-graph inlining | Not directly affected | Call-graph analysis concerns function relationships, unrelated to instanceof narrowing | |
||||
| #5 Range inference | Not directly affected | Integer ranges are unrelated to OOP inheritance | |
||||
|
||||
### Key Conclusion |
||||
|
||||
Although `instanceof` narrowing is not sufficient to deterministically devirtualize all calls, it provides the guarantee of **type member visibility**, which can eliminate method lookups, improve error detection, and generate more precise C++ type signatures (`Foo*` vs `php::Var`). |
||||
|
||||
The information that can truly drive devirtualization comes from: |
||||
1. **Final classes** — provided by source declarations |
||||
2. **Whole-program class hierarchy analysis** — after compiling all code, one knows which classes are de facto final (no subclasses in the codebase) |
||||
3. **Call-site-specific type inference** — tracking from `new Foo()` assignment to the call site |
||||
|
||||
--- |
||||
|
||||
## Conclusion |
||||
|
||||
Under the GCC/Clang secondary compilation architecture, the AOT compiler's optimization strategy should be: |
||||
|
||||
1. **"Freeze" PHP's dynamic type information into static C++ types** — this is what GCC cannot do itself |
||||
2. **"Land" virtual calls as direct calls** — so GCC can inline |
||||
3. **"Promote" heap allocation to stack allocation** — so GCC can perform SROA |
||||
4. **Do not compete with GCC on SSA-level optimization** — SCCP, DCE, CSE are left to GCC |
||||
5. **Focus on cross-translation-unit information** — call graph, class hierarchy — which even LTO has difficulty covering |
||||
@ -0,0 +1,569 @@ |
||||
# HHVM/HHBBC Compiler Review: Whole-Program Analysis, Type System, and Optimization Pipeline |
||||
|
||||
> 2026-06-11 · Based on source review of /home/swoole/workspace/cpp/hhvm |
||||
|
||||
--- |
||||
|
||||
## 1. Architecture Overview |
||||
|
||||
HHVM's compilation system is divided into three layers: |
||||
|
||||
| Layer | Component | Responsibility | |
||||
|------|------|------| |
||||
| Frontend | HackC (hphp/hack) | Hack/PHP source → HHAS (Hack Assembly) | |
||||
| Middle end | HHBBC (hphp/hhbbc) | HHBC bytecode → optimized HHBC bytecode | |
||||
| Backend | JIT (hphp/runtime/vm/jit) | HHBC → x86-64 machine code (runtime tracing JIT) | |
||||
|
||||
HHBBC (~80k lines of C++) is a **whole-program bytecode optimizer** based on fixed-point iterative analysis. Core files: |
||||
|
||||
| File | Lines | Purpose | |
||||
|------|------|------| |
||||
| `interp.cpp` | 6,685 | Abstract interpreter (forward dataflow) | |
||||
| `index.cpp` | 30,132 | Whole-program index: type info, dependency tracking, incremental re-analysis | |
||||
| `type-system.h` | 1,610 | Type lattice definition (trep + specialization) | |
||||
| `type-system.cpp` | 8,451 | Type operation implementation (meet/join/subtype/union) | |
||||
| `dce.cpp` | 3,102 | Type-aware dead code elimination (local + global) | |
||||
| `analyze.cpp` | 2,293 | Function-level dataflow analysis driver | |
||||
| `optimize.cpp` | 962 | Type-aware optimization pass | |
||||
| `cfg-opts.cpp` | ~300 | CFG optimization (unreachable block removal, exception edge simplification) | |
||||
|
||||
Comparison with AOT Compiler: |
||||
|
||||
| Dimension | HHBBC | AOT Compiler | |
||||
|------|-------|-------------| |
||||
| Compilation target | HHBC → optimized HHBC → JIT machine code | PHP → C++ → binary | |
||||
| Analysis level | Bytecode-level (whole-program) | AST-level (single-file) | |
||||
| IR form | php::Func (factored CFG + Bytecode) | PHP-Parser AST | |
||||
| Type inference | trep + specialization + fixed-point iteration | SSA + manual type annotations | |
||||
| Optimization scope | Whole-program (cross-file/cross-function) | Single function + class hierarchy | |
||||
| Call graph | Dynamically built (Index + assumption) | Static (classExtends + classMethodOverride) | |
||||
| Concurrency | Function-level parallel analysis | Serial execution | |
||||
| Runtime | HHVM (JIT + refcounting GC) | phpx (RAII wrapper around Zend API) | |
||||
| Code size | ~80k lines of C++ (HHBBC only) | ~15k lines of PHP | |
||||
|
||||
--- |
||||
|
||||
## 2. Reusable Innovative Designs |
||||
|
||||
### 2.1 Whole-Program Fixed-Point Analysis |
||||
|
||||
**Files**: `hphp/hhbbc/README`, `analyze.cpp`, `index.cpp` |
||||
|
||||
HHBBC's core algorithm: iterative convergence analysis across the **whole program**. |
||||
|
||||
``` |
||||
Algorithm flow: |
||||
1. Initialize work list = all functions/classes |
||||
2. Analyze each work unit in parallel (read-only Index, producing a new result) |
||||
3. Single-threaded: merge the new result into the Index |
||||
4. If the Index was updated → add back functions that depend on that info to the work list |
||||
5. Repeat until the Index reaches a fixed point (info no longer changes) |
||||
6. Final parallel optimization pass |
||||
``` |
||||
|
||||
Key design principles: |
||||
- Info in the Index can only **shrink** (shrinking types: becoming more precise) |
||||
- Types within function analysis can only **grow** (growing types: accumulated within the analysis context) |
||||
- Index info is **never wrong** (soundness guarantee) — it can only be insufficiently precise |
||||
|
||||
**AOT adoption priority: P0** |
||||
|
||||
Currently AOT is single-file serial compilation with no cross-file/cross-function analysis. What can be adopted: |
||||
|
||||
1. **Index structure**: store inferred results such as each function's return type, parameter types, and property types |
||||
2. **Dependency tracking**: record every query ("function F queried the return type of method M of class C") and add F back to the work list when C::M's type is updated |
||||
3. **Iterative convergence**: initialize all functions as "unknown" → analyze round by round → until type info stops changing |
||||
|
||||
Implementation suggestion: |
||||
```php |
||||
// Built in Preprocessor |
||||
class AnalysisIndex { |
||||
// Function return types (shrinking across iterations) |
||||
public array $returnTypes = []; // funcName => TypeInfo |
||||
|
||||
// Dependency graph: funcName => [depended-on funcName, ...] |
||||
public array $dependencies = []; |
||||
|
||||
// Function parameter states (collected from call sites) |
||||
public array $paramTypes = []; // funcName => [argIdx => TypeInfo] |
||||
|
||||
// Public static property states |
||||
public array $publicStaticProps = []; // className::prop => TypeInfo |
||||
} |
||||
``` |
||||
|
||||
Then repeatedly analyze functions in topological order until `$returnTypes` and `$paramTypes` stop changing. |
||||
|
||||
--- |
||||
|
||||
### 2.2 Trep Type Lattice (Bitset Type Lattice) |
||||
|
||||
**Files**: `hphp/hhbbc/type-system.h`, `type-system-bits.h`, `type-system-detail.h` |
||||
|
||||
HHBBC's type system is the cornerstone of the entire optimizer and is ingeniously designed: |
||||
|
||||
**Base mechanism — trep (type representation):** |
||||
```cpp |
||||
// Each "base type" is a bit |
||||
BUninit, BInitNull, BFalse, BTrue, BInt, BDbl, |
||||
BCls, BLazyCls, BFunc, BClsMeth, BEnumClassLabel, |
||||
BObj, BRes, BRFunc, BRClsMeth, |
||||
BSStr, BCStr, // uncounted/counted strings |
||||
BSVec, BCVec, BSVecE, BCVecE, BSVecN, BCVecN, // arrays' counted × empty dimensions |
||||
// ... Dict, Keyset follow the same pattern |
||||
``` |
||||
|
||||
Types are represented as union types via bitset combinations. For example, `Int|String` is `BInt|BStr`. |
||||
|
||||
**Specialization — extra info attached to types:** |
||||
``` |
||||
Int=n — known constant integer value |
||||
Dbl=n — known constant float value |
||||
{S,C}Str=s — known constant string value |
||||
Obj{<}=c — known class type (exact or subclass) |
||||
Arr(T1,T2,...) — known array shape (packed array) |
||||
Arr([T1:T2]) — known array key/value types |
||||
``` |
||||
|
||||
**Key innovations:** |
||||
- `counted` / `uncounted` dimension: strings and arrays distinguish whether they are reference-counted, allowing the compiler to optimize uncounted (static/constant) values more aggressively |
||||
- `empty` / `non-empty` dimension: arrays distinguish empty/non-empty, eliminating redundant empty checks |
||||
- Monotonic lattice: types can only change in one direction (Index types can only shrink, analysis types can only grow) |
||||
|
||||
**AOT adoption priority: P0** |
||||
|
||||
Currently AOT uses discrete constants like `TYPE_INT`, `TYPE_STRING`, `TYPE_ARRAY` with no fine-grained info such as union types or uncounted markers. It could: |
||||
|
||||
1. Use bitsets to represent union types (`int|string` instead of `mixed`) |
||||
2. Add an `Immutable` marker (values provably immutable at compile time) |
||||
3. Add a `NonEmpty` marker (arrays provably non-empty at compile time) |
||||
4. Build a type lattice defining meet (⊓) and join (⊔) operations |
||||
|
||||
```php |
||||
// Current |
||||
public const TYPE_INT = 1; |
||||
|
||||
// Suggested: bitset |
||||
class Type { |
||||
const BINT = 1 << 0; |
||||
const BSTRING = 1 << 1; |
||||
const BBOOL = 1 << 2; |
||||
const BFLOAT = 1 << 3; |
||||
// ... |
||||
const BIMMUTABLE = 1 << 16; // immutable marker |
||||
const BNONEMPTY = 1 << 17; // non-empty marker |
||||
|
||||
// union type: int|string = BINT | BSTRING |
||||
// meet: narrower = more specific |
||||
// join: wider = more general |
||||
} |
||||
``` |
||||
|
||||
--- |
||||
|
||||
### 2.3 Abstract Interpreter |
||||
|
||||
**Files**: `hphp/hhbbc/interp.cpp` (6,685 lines), `interp.h` |
||||
|
||||
HHBBC uses an **abstract interpreter** for function-level type inference. This differs from traditional SSA + constraint solving: |
||||
|
||||
``` |
||||
Algorithm (analyze_func): |
||||
1. Initialize the entry block's input state (parameter types = conservative assumptions from the Index) |
||||
2. Work list ← entry blocks |
||||
3. While work list not empty: |
||||
a. Pop a block |
||||
b. Run the abstract interpreter instruction-by-instruction over the block |
||||
c. On instructions that may throw → propagate current state to the exception edge |
||||
d. On branch instructions → propagate the post-branch state to the taken edge |
||||
e. Block may fallthrough → propagate final state to the fallthrough edge |
||||
f. If the target block's input state changed → add to work list |
||||
``` |
||||
|
||||
Key characteristics of the abstract interpreter: |
||||
- **State propagation**: compute a new `State` (locals + eval stack type info) after each instruction |
||||
- **Factored CFG**: exception edges are modeled separately (`FactoredExitBlock`), so ordinary instructions don't break basic blocks just because they "may throw" |
||||
- **Type specialization**: branch conditions automatically narrow types (e.g. `if (is_int($x))` → `$x` narrowed to `Int` in the then branch) |
||||
- **Constant propagation**: built-in constant folding and value tracking |
||||
|
||||
**AOT adoption priority: P1** |
||||
|
||||
Currently AOT's `SsaTypeOptimizer` does some type narrowing, but: |
||||
- No factored CFG concept (exceptions don't participate in dataflow) |
||||
- No abstract interpreter framework (each instruction defines `step(state) → new_state`) |
||||
- No iterative dataflow analysis |
||||
|
||||
A lightweight abstract interpretation can be implemented on top of SSA: |
||||
```php |
||||
class AbstractInterpreter { |
||||
// Run abstract interpretation over each SSA basic block |
||||
function analyzeBlock(Block $block, State $inputState): State { |
||||
foreach ($block->instructions as $instr) { |
||||
$inputState = $this->step($instr, $inputState); |
||||
} |
||||
return $inputState; |
||||
} |
||||
|
||||
// Each instruction defines how to transform state |
||||
function step(Instruction $instr, State $state): State { |
||||
// switch on instruction type |
||||
// return new State with updated types |
||||
} |
||||
} |
||||
``` |
||||
|
||||
--- |
||||
|
||||
### 2.4 Type-Aware Dead Code Elimination |
||||
|
||||
**File**: `hphp/hhbbc/dce.cpp` (3,102 lines) |
||||
|
||||
HHBBC's DCE differs from traditional liveness-based DCE — it **combines type analysis** to discover more dead code: |
||||
|
||||
``` |
||||
Two kinds of DCE: |
||||
1. Local DCE — within a single basic block |
||||
- Traverse the block backward |
||||
- Maintain a "reverse stack": mark which eval stack slots will be used in the future |
||||
- Unused stack slots → the instruction producing that slot can be deleted |
||||
- Unused local stores → eliminated |
||||
|
||||
2. Global DCE — across basic blocks |
||||
- Liveness analysis over locals |
||||
- Allows eliminating dead stores across blocks |
||||
``` |
||||
|
||||
Key design: **type awareness** — DCE needs to know each instruction's type to judge correctly. For example: |
||||
- If `$x` has type `Bottom` (unreachable) on some path, code using `$x` may be dead code |
||||
- If `$x` is a counted type, related inc/dec ref operations cannot be eliminated |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
Currently AOT's DCE essentially relies on GCC/Clang's `-O2`. Compile-time local DCE can be added: |
||||
- Eliminate assignments to unused local variables |
||||
- Eliminate side-effect-free pure computations (if the result is unused) |
||||
- Use type info to determine whether an operation may have side effects |
||||
|
||||
--- |
||||
|
||||
### 2.5 DataType Encoding: 3-of-7 Error-Correcting Code |
||||
|
||||
**File**: `hphp/runtime/base/datatype.h` |
||||
|
||||
HHVM's runtime type tags use an extremely clever bit encoding: |
||||
|
||||
```cpp |
||||
// DataType is uint8_t |
||||
// - bit 0 (LSB): countedness — 0 = definitely uncounted |
||||
// - bits 1-7: 3-of-7 error-correcting code — exactly 3 bits set to 1 |
||||
|
||||
// Type detection becomes simple bit operations: |
||||
// Check if Vec or Dict: dt <= KindOfVec |
||||
// Check if it has a persistent version: dt <= KindOfString |
||||
// Check if null/uninit: dt >= KindOfUninit |
||||
``` |
||||
|
||||
Characteristics of the 3-of-7 encoding: |
||||
- There are exactly C(7,3) = 35 8-bit values with exactly 3 bits set to 1 (each persistent/counted pair shares the same 3-of-7 code) |
||||
- Detecting any type: `(dt & type_mask) == type_tag` completes in two instructions |
||||
- Unsigned LT/GT comparisons implement efficient type-group detection |
||||
|
||||
**AOT adoption priority: P4** |
||||
|
||||
This informs the **quality of generated code** for the AOT compiler — but mainly at the phpx layer. Consider giving phpx's `Variant` type tags a more efficient encoding to optimize runtime type checks like `is_int()`/`is_string()`. |
||||
|
||||
--- |
||||
|
||||
### 2.6 RepoAuthType: Space-Efficient Type Storage in Bytecode |
||||
|
||||
**Files**: `hphp/runtime/base/repo-auth-type.h`, `repo-auth-type-tags.h` |
||||
|
||||
HHBBC encodes the type info derived from analysis as `RepoAuthType`, embeds it in the bytecode stream (`AssertRAT` instruction), and makes it available to the JIT. |
||||
|
||||
Design points: |
||||
- Compact encoding (`CompactTaggedPtr`): type tag + optional pointer (class name/array shape) packed into one pointer width |
||||
- Covers the complete lattice from `Uninit` (most precise) to `Cell` (most general) |
||||
- `SubObj` / `SubCls` tags support subclass relationships |
||||
- Array shape specialization (precise type of packed arrays) |
||||
|
||||
**AOT adoption priority: P3** |
||||
|
||||
Currently AOT's type annotations are represented through the C++ type system (`int64_t`, `php::string`, `php::array`). For typed properties, RAT-like thinking can be used to generate more precise C++ type declarations. |
||||
|
||||
--- |
||||
|
||||
### 2.7 Index Dependency Tracking |
||||
|
||||
**File**: `hphp/hhbbc/index.cpp` (30,132 lines) |
||||
|
||||
The Index is not just storage for type info — more centrally it provides the **dependency tracking mechanism**: |
||||
|
||||
``` |
||||
Dependency kinds (DependencyKind): |
||||
- ReturnTy — function return type |
||||
- ConstVal — constant value |
||||
- ClsConst — class constant |
||||
- PropType — property type |
||||
- PublicSProp — public static property type (especially important!) |
||||
``` |
||||
|
||||
When a function's return type is updated in the Index, all functions that queried that return type are marked for re-analysis. |
||||
|
||||
**Special handling of public static properties**: public static properties can be modified by any function, so the Index tracks all mutation operations. When analysis finds that a static property is never modified, more aggressive constant propagation becomes possible. |
||||
|
||||
**AOT adoption priority: P1** |
||||
|
||||
Complements the whole-program analysis in 2.1 — Index dependency tracking is the foundation for implementing iterative analysis. In AOT: |
||||
|
||||
```php |
||||
class Index { |
||||
// Return types: funcName => Type |
||||
// Dependencies: funcName => [depends_on_funcName, ...] |
||||
// Dirty flags: funcName => bool (needs re-analysis) |
||||
} |
||||
``` |
||||
|
||||
After Preprocessor scans all files, build the initial Index, then iterate until convergence. |
||||
|
||||
--- |
||||
|
||||
### 2.8 Factored CFG (Exception-Factored Control Flow Graph) |
||||
|
||||
**Files**: `hphp/hhbbc/cfg.h`, `parse.cpp` |
||||
|
||||
HHBBC's control flow graph **factors out** exception edges: it doesn't terminate basic blocks at every instruction that may throw, instead letting basic blocks be as large as possible. |
||||
|
||||
``` |
||||
Traditional CFG: |
||||
instr1 ; block 1 |
||||
instr2(may_throw) ; block 1 terminates here (because it may throw) |
||||
--- |
||||
instr3 ; block 2 |
||||
|
||||
Factored CFG: |
||||
instr1 ; block 1 |
||||
instr2(may_throw) |
||||
instr3 |
||||
; block 1 contains multiple instructions |
||||
; exception edges connect from the factored exit edge to the exception handler |
||||
``` |
||||
|
||||
Benefits: |
||||
- Larger basic blocks → more efficient dataflow analysis (fewer block boundaries) |
||||
- Type info can rule out exception possibilities → exception edges can be deleted during optimization |
||||
- Easier instruction scheduling during the JIT stage |
||||
|
||||
**AOT adoption priority: P4** |
||||
|
||||
For an AOT compiler (translating to C++ rather than directly generating machine code), the CFG is mostly handled by GCC. But for function coloring and SSA analysis, more precise exception modeling can be a reference. |
||||
|
||||
--- |
||||
|
||||
### 2.9 Parallel Analysis |
||||
|
||||
**Files**: `hphp/hhbbc/parallel.cpp` (69 lines), `README` |
||||
|
||||
In HHBBC's fixed-point iteration, the analysis of each work unit can be **fully parallel**: |
||||
|
||||
``` |
||||
Thread-safety model: |
||||
- Analysis phase: can only read the Index (internally thread-safe) + read php metadata (immutable) |
||||
- Merge phase: single-threaded Index update (no lock needed) |
||||
``` |
||||
|
||||
This leverages the "Index info is never wrong" property — even if two threads analyze based on different versions of the Index, merging results never produces wrong info. |
||||
|
||||
**AOT adoption priority: P3** |
||||
|
||||
Similar to KPHP's pipeline parallelism. In AOT, independent functions/classes can be analyzed in parallel. Note that Index merging must be serial (or use lock-free structures). |
||||
|
||||
--- |
||||
|
||||
### 2.10 Public Static Property Optimization |
||||
|
||||
**File**: `hphp/hhbbc/index.cpp` |
||||
|
||||
HHBBC tracks the mutation of every public static property across the whole program: |
||||
|
||||
``` |
||||
- Initial state: conservative assumption (may be modified by any function) |
||||
- During each analysis round, record which functions modify which static props |
||||
- If a static prop is never modified during analysis → can be constant-folded |
||||
- If a static prop is only ever assigned one type → its type can be narrowed |
||||
``` |
||||
|
||||
This is more precise than a simple "is it written" analysis because it is a **whole-program** analysis that can see cross-file mutations. |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
Currently in AOT, public static properties always use `Variant` (mixed type). Through whole-program analysis, static properties that are only assigned internally can be optimized to precise C++ types. |
||||
|
||||
--- |
||||
|
||||
## 3. Toolchain Analysis |
||||
|
||||
### 3.1 Test Infrastructure |
||||
|
||||
HHVM has a massive test system: |
||||
|
||||
| Layer | Directory | Count | Purpose | |
||||
|------|------|------|------| |
||||
| Quick test | `hphp/test/quick/` | 865 .php files | Fast regression tests | |
||||
| Slow test | `hphp/test/slow/` | 7,927 .php files | Comprehensive functional/performance tests | |
||||
| Zend test | `hphp/test/zend/` | ~4,500 | PHP compatibility tests (from php-src) | |
||||
| Ext test | `hphp/test/ext/` | ~800 | Extension feature tests | |
||||
| Server test | `hphp/test/server/` | ~100 | HTTP/RPC integration tests | |
||||
| HHBBC unit test | `hphp/hhbbc/test/` | 3 C++ files | Compiler internal tests | |
||||
| **Total** | | **~14,675** | | |
||||
|
||||
Test runner characteristics: |
||||
- Quick vs Slow layering: Quick (~1 second to run) for pre-commit, Slow (~10 minutes) for CI |
||||
- Supports multiple run modes: interp / JIT / hhbbc + JIT / RepoAuthoritative |
||||
- Zend tests: directly reuse php-src's official tests to verify PHP compatibility |
||||
|
||||
**AOT adoption:** |
||||
- Quick/Slow layered test strategy — classify the existing `tests/compiler/` by run time |
||||
- Directly reuse php-src's official PHPT tests — verify the AOT compiler's PHP behavior compatibility |
||||
- HHBBC has too few internal unit tests (only 3) — should not be emulated; AOT's PHPUnit coverage is better |
||||
|
||||
### 3.2 Hack Type Checker |
||||
|
||||
HHVM's Hack language has a complete **static type checker** (`hphp/hack/`) that runs independently of the compiler: |
||||
|
||||
- Compile-time type annotations (`int`, `string`, `vec<T>`, `dict<TK,TV>`, `shape(...)`) |
||||
- Gradual typing: can migrate gradually from unannotated code |
||||
- IDE integration (LSP protocol support) |
||||
- Type coverage tracking |
||||
|
||||
**AOT adoption:** |
||||
- Currently AOT uses annotations like `@phpstan-type`; phpstan can be integrated for pre-compilation type checking |
||||
- Type coverage is a useful metric: X% of functions/variables have precise type annotations |
||||
|
||||
### 3.3 Tracing / Debug Infrastructure |
||||
|
||||
HHVM has rich debugging and tracing: |
||||
- `TRACE_SET_MOD(hhbbc)` — module-level conditional logging (compile-time switch) |
||||
- `hphp/tools/` — various analysis tools (bytecode viewer, profiler, etc.) |
||||
- `debug.cpp` — human-readable printing of types/states |
||||
|
||||
**AOT adoption:** |
||||
- The current `-vv` verbose output can adopt the TRACE_MODULE approach to filter logs by module |
||||
- Visualization of type analysis results can aid debugging the optimizer |
||||
|
||||
### 3.4 RepoAuthoritative Mode |
||||
|
||||
HHVM supports a **compile once, deploy many times** mode: |
||||
|
||||
``` |
||||
Source → HHBBC whole-program analysis → optimized bytecode Repo → multiple processes load the Repo directly |
||||
``` |
||||
|
||||
- The Repo stores **pre-analyzed type info** (RepoAuthType) |
||||
- No re-inference of types at runtime |
||||
- Bytecode-level interning (string/class/function ids) |
||||
- Multiple processes share the same Repo (mmap) |
||||
|
||||
**AOT adoption: P4** |
||||
|
||||
Currently AOT directly generates `.cc` files and compiles them into a binary, already on the "compile once, deploy many times" path. But the Repo's mmap sharing idea can be used for constant pool sharing in multi-process environments (rather than each process loading independently). |
||||
|
||||
--- |
||||
|
||||
## 4. Type System Compatibility Analysis |
||||
|
||||
### 4.1 Hack vs PHP vs AOT Compiler |
||||
|
||||
| Feature | PHP 8.2 | Hack | AOT Compiler | |
||||
|------|---------|------|-------------| |
||||
| Base types | mixed, int, string, float, bool, array, null, void, never | int, string, float, bool, null, void, noreturn, mixed, dynamic, nonnull, nothing | Same as PHP | |
||||
| Union types | `int\|string` | `int\|string` (but discouraged in practice) | Supported | |
||||
| Intersection types | `X&Y` (8.1+) | Via `where` constraints | Not supported | |
||||
| Generics | None | `vec<T>`, `dict<TK,TV>`, `class Box<T>` | `std::vector<T>` (native C++) | |
||||
| Array types | `array` | `vec<T>`, `dict<TK,TV>`, `keyset<T>` | `php::array` | |
||||
| Shapes | None | `shape('x' => int, 'y' => string)` | None (object suggested) | |
||||
| Enums | enum (8.1) | enum, enum class (with label) | Supports PHP 8.1 enum | |
||||
| Nullable types | `?int` | `?int` (function params only) | Supported | |
||||
| nothing / bottom | None | Yes (empty function return, unreachable) | None | |
||||
| dynamic | None | Yes (selectively skip type checking) | None | |
||||
|
||||
### 4.2 Key Differences in the Type Lattice |
||||
|
||||
Hack/HHBBC's type system is far richer than PHP's: |
||||
|
||||
``` |
||||
HHBBC type lattice: |
||||
|
||||
Cell (any value) |
||||
| |
||||
InitCell (not Uninit) |
||||
| | |
||||
Prim Boxed types (Obj, Res, ...) |
||||
| |
||||
InitPrim (not null) |
||||
| | | |
||||
Num Bool Str/ArrKey |
||||
| | |
||||
Int Dbl |
||||
|
||||
Bottom: Bottom (no value - unreachable) |
||||
Top: Cell (any value) |
||||
``` |
||||
|
||||
Characteristics: |
||||
- **Bottom**: represents the type of unreachable paths, enabling dead code elimination |
||||
- **Counted/Uncounted**: static strings vs runtime-allocated strings, different lifetimes |
||||
- **Array shapes**: `dict<'name' => string, 'age' => int>` is a distinct type |
||||
- **Wait handle**: `WaitH<T>` represents the type of async results |
||||
|
||||
### 4.3 Hack's Gradual Typing Design |
||||
|
||||
Hack evolved from PHP and supports gradual migration: |
||||
- `mixed` — any type (equivalent to unannotated PHP) |
||||
- `dynamic` — any type, and no type errors reported (a looser mixed) |
||||
- `<<__Soft>>` — soft type hints (not enforced at runtime) |
||||
|
||||
This design allows large codebases to add type annotations incrementally, avoiding an "all or nothing" approach. |
||||
|
||||
### 4.4 Syntax Differences from the AOT Compiler |
||||
|
||||
| Feature | HHBBC Input (HHAS) | AOT Compiler | |
||||
|------|------|-------------| |
||||
| Base PHP version | Hack (PHP 5.6 branch) | PHP 8.2+ | |
||||
| Type annotations | Mandatory (required by Hack) | Optional (phpstan annotations) | |
||||
| Generics | `vec<T>`, `dict<K,V>`, custom generic classes | No native generics | |
||||
| Lambda / closures | `$x ==> $x + 1` (short lambda) | PHP closures (`function($x) { return $x + 1; }`) | |
||||
| async / await | Native support (WaitHandle) | None | |
||||
| Shapes | `shape('x' => int)` | None | |
||||
| Enum class | Supported (enum class with label) | PHP 8.1 enum only | |
||||
| XHP | HTML template syntax | None | |
||||
| Case types | `case type T = int \| string` | None | |
||||
|
||||
--- |
||||
|
||||
## 5. Summary and Priority Recommendations |
||||
|
||||
| Priority | Technique | Difficulty | Benefit | Notes | |
||||
|--------|------|------|------|------| |
||||
| **P0** | Whole-program fixed-point analysis | Very high | Very high | Cross-file type inference, return type narrowing, eliminating pseudo-dynamic calls | |
||||
| **P0** | Trep type lattice | Medium | Very high | Precise union types, uncounted markers, non-empty markers — improves precision of all optimizations | |
||||
| **P1** | Abstract interpreter | High | High | Replaces/enhances current SSA analysis, supports branch type narrowing and constant propagation | |
||||
| **P1** | Index dependency tracking | High | High | Foundation for whole-program analysis, incremental compilation | |
||||
| **P2** | Type-aware DCE | Medium | Medium | Eliminate more dead code, reduce generated C++ code size | |
||||
| **P2** | Public static prop optimization | Medium | Medium | Precise typing of global static properties | |
||||
| **P3** | RepoAuthType storage | Low | Medium | Embed precise type info in artifacts (limited value for the C++ generation stage) | |
||||
| **P3** | Parallel analysis | High | Medium | Faster compilation of large projects | |
||||
| **P4** | DataType encoding optimization | Low | Low | Micro-optimization of phpx Variant type checks | |
||||
| **P4** | Factored CFG | High | Low | C++ compiler already handles CFG optimization | |
||||
| **P5** | Repo mmap sharing | High | Low | Optimization for specific deployment scenarios | |
||||
|
||||
### Key Takeaways |
||||
|
||||
1. **HHBBC's whole-program analysis is its biggest differentiator** — this is the AOT compiler's greatest architectural gap. Single-file analysis cannot see cross-file type info, forcing many calls to use dynamic dispatch. |
||||
|
||||
2. **The type system is the core engine of optimization** — HHBBC's trep + specialization + monotonic lattice is a huge investment (about 12k lines), but it is the quality foundation for all optimization passes. |
||||
|
||||
3. **Dependency tracking is the key to incremental analysis** — the Index automatically records "who queried what", so only affected functions need re-analysis, avoiding re-analyzing the whole program each time. |
||||
|
||||
4. **Factored CFG and DataType encoding are more JIT-oriented** — these designs serve runtime JIT optimization. The AOT compiler generates C++ source, and GCC/Clang handle these low-level optimizations, so the investment should not be duplicated. |
||||
|
||||
5. **The test infrastructure is worth learning from** — Quick/Slow layering, directly reusing php-src tests, and multiple run modes all have direct reference value for building the AOT compiler's test CI. |
||||
@ -0,0 +1,451 @@ |
||||
# KPHP Compiler Review: Design, Optimization, Toolchain, and Syntax Compatibility Analysis |
||||
|
||||
> 2026-06-11 · Based on source review of /home/swoole/workspace/cpp/kphp |
||||
|
||||
--- |
||||
|
||||
## 1. Architecture Overview |
||||
|
||||
| Dimension | KPHP | AOT Compiler | |
||||
|------|------|-------------| |
||||
| Compilation target | PHP → C++ → binary | PHP → C++ → binary | |
||||
| IR form | Custom vertex (op_*) tree | PHP-Parser AST → C++ string | |
||||
| Type inference | Iterative convergence graph inference (converging toward generalization) | SSA + manual type annotations | |
||||
| Intermediate optimization | Multi-pass AST rewriting (~60+ pipes) | Direct AST → C++ translation + a few optimizations | |
||||
| Runtime | Self-developed (allocator/string/array/mixed) | phpx (C++ RAII wrapper around Zend API) | |
||||
| Concurrency model | Single thread + reactor/epoll | Depends on Zend/TSRM | |
||||
| Thread safety | None (lock-free memory allocator) | Zend TSRM | |
||||
| Code size (compiler) | ~18k lines of pipe code | ~6k lines of CompilerBase | |
||||
|
||||
--- |
||||
|
||||
## 2. Reusable Innovative Designs |
||||
|
||||
### 2.1 Rewrite Rules DSL (Pattern-Matching Optimization Rules) |
||||
|
||||
**File**: `compiler/rewrite-rules/early_opt.rules` |
||||
|
||||
A declarative DSL describes AST rewriting and generates C++ optimization code at compile time. The rule format is `(pattern) => (replacement)`, supporting conditional clauses and embedded C++ expressions: |
||||
|
||||
```lisp |
||||
;; strlen constant folding |
||||
(op_func_call {"strlen"} arg:(op_string)) |
||||
=> (op_int_const { std::to_string(arg->str_val.size()) }) |
||||
|
||||
;; explode index direct access → specialized version |
||||
(op_index (op_func_call {"explode"} delim s) k:(op_int_const)) |
||||
=> (op_func_call {"_explode_nth"} delim s k) |
||||
|
||||
;; ("" . $x) → (string)$x — eliminate meaningless concatenation |
||||
(op_concat (op_string {""}) x) => (op_conv_string x) |
||||
|
||||
;; substring type optimization: conv(substr(...)) → conv(_tmp_substr(...)) |
||||
(op_conv_int x) if let x2 { to_tmp_string_expr(x) } => (op_conv_int x2) |
||||
``` |
||||
|
||||
**AOT adoption priority: P0** |
||||
|
||||
Currently `FuncCallOptimizer`'s `strlen`/`count` optimizations are hardcoded. Introducing a similar rule engine can: |
||||
- Add optimizations declaratively, reducing maintenance cost |
||||
- Use `if let` conditions for local pattern variable binding |
||||
- Separate rule files from the compiler, making hot reloading feasible |
||||
|
||||
Implementation suggestion: implement a `RewriteRule` class at the PHP level and load/match it in `FuncCallOptimizer`. |
||||
|
||||
--- |
||||
|
||||
### 2.2 Smart instanceof / Smart Casts (Type Narrowing) |
||||
|
||||
**File**: `compiler/pipes/transform-to-smart-instanceof.cpp` |
||||
|
||||
After `if ($x instanceof A)`, `$x` is automatically renamed to `instance_cast<A>($x)` inside the if body. The core innovation is doing variable splitting **before type inference**: |
||||
|
||||
```php |
||||
// PHP source |
||||
if ($x instanceof A) { |
||||
$x->methodOfA(); // $x automatically becomes instance_cast<A>($x) |
||||
} |
||||
|
||||
// Reverse guard pattern |
||||
if (!($x instanceof A)) return; |
||||
// After this, $x is replaced by instance_cast<A>($x) across the whole function scope |
||||
``` |
||||
|
||||
It also handles the renaming of same-named variables across different `catch (SomeClass $e)` blocks to prevent assumption confusion. |
||||
|
||||
**AOT adoption priority: P0** |
||||
|
||||
Currently `SsaTypeOptimizer` only narrows int/float/string base types. Object type narrowing can be added: |
||||
1. Identify `instanceof` guards in the SSA builder |
||||
2. Replace with the target subclass type in the then/else branches |
||||
3. Combine with the existing `stableObjects` mechanism for devirtualization |
||||
|
||||
--- |
||||
|
||||
### 2.3 Pipeline Parallel Compilation |
||||
|
||||
**File**: `compiler/compiler.cpp` |
||||
|
||||
The compilation process works at function granularity, chaining pipes via `operator>>` and processing them in parallel across multiple threads: |
||||
|
||||
```cpp |
||||
SchedulerConstructor{scheduler} |
||||
>> PipeC<LoadFileF>{} |
||||
>> PipeC<FileToTokensF>{} |
||||
>> PipeC<ParseF>{} |
||||
>> PassC<GenTreePostprocessPass>{} |
||||
/* ... 60+ pipes */; |
||||
``` |
||||
|
||||
Three pipe types: |
||||
- **PipeC\<T\>**: general transformation, input→output |
||||
- **PassC\<T\>**: function-level transformation, traverses all AST vertices |
||||
- **SyncC\<T\>**: synchronization point, outputs only after all inputs are processed |
||||
|
||||
Different functions can be processed simultaneously at different stages; global storage uses thread-safe or lock-free structures. |
||||
|
||||
**AOT adoption priority: P3** |
||||
|
||||
Currently `Preprocessor` → `CompilerBase` is serial. For large projects, function-level parallelism can be introduced: |
||||
- Compile classes/functions independently at their granularity |
||||
- Use `SyncC` synchronization points to merge global symbol tables |
||||
|
||||
--- |
||||
|
||||
### 2.4 Switch Splitting (State Machine Transformation) |
||||
|
||||
**File**: `compiler/pipes/split-switch.cpp` |
||||
|
||||
Extract each case branch of a switch into an independent function, driven by a state variable: |
||||
|
||||
```cpp |
||||
// Each case becomes: |
||||
int case_state = 0; |
||||
auto case_res = switch_func_N(&case_state); |
||||
if (case_state == 1) return case_res; // normal return |
||||
if (case_state == -1) break; // break semantics |
||||
``` |
||||
|
||||
`break N` and `continue N` are converted to setting the state variable to `-1` + return, consistent with the `_brk_flag` / `_cnt_flag` approach previously implemented in AOT. |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
Large switches can be split into independent functions, reducing single-function complexity and giving GCC more room for inlining/optimization. |
||||
|
||||
--- |
||||
|
||||
### 2.5 Constant Immutability Markers and init-once |
||||
|
||||
**Files**: `compiler/pipes/collect-const-vars.cpp`, `runtime-common/core/memory-resource/` |
||||
|
||||
Compile-time constant arrays/strings use special refcount markers: |
||||
|
||||
```cpp |
||||
ExtraRefCnt::for_global_const // immutable, triggers COW on modification |
||||
ExtraRefCnt::for_instance_cache // shared across requests, not modifiable |
||||
``` |
||||
|
||||
These constants are stored in the data section, initialized once at server startup, and used read-only in subsequent requests. Any modification automatically triggers COW. |
||||
|
||||
**AOT adoption priority: P1** |
||||
|
||||
Currently AOT already promotes constant arrays to static variables, but a finer-grained immutability marker mechanism can be introduced to reduce unnecessary COW copies (when the compiler can prove a variable is never modified). |
||||
|
||||
--- |
||||
|
||||
### 2.6 Function Specialization (Multi-Version Generation) |
||||
|
||||
**File**: `compiler/pipes/early-optimization.cpp` |
||||
|
||||
Functions are specialized by their arguments before type inference: |
||||
|
||||
- `microtime()` → `_microtime_float()` or `_microtime_string()` (based on the true/false argument) |
||||
- `list() + explode()` → `_explode_tupleN()` (precise N-tuple type) |
||||
- `explode()[N]` → `_explode_nth()` (O(1) direct access to the Nth element) |
||||
- `substr()` in a function argument position → `_tmp_substr()` (avoid string copying) |
||||
|
||||
The key is that **specialized versions return more precise types**. For example, `microtime()` returns `mixed`, while `_microtime_float()` returns `float`. |
||||
|
||||
**AOT adoption priority: P1** |
||||
|
||||
`FuncCallOptimizer` currently only does constant folding; it can be extended to **multi-version specialization**: |
||||
|
||||
```php |
||||
// Current |
||||
$result = strlen($s); // returns mixed/int |
||||
|
||||
// After optimization |
||||
$result = _strlen_string($s); // compile-time-determined int return |
||||
``` |
||||
|
||||
--- |
||||
|
||||
### 2.7 Class Assumptions: A Priori Type Prediction |
||||
|
||||
**File**: `compiler/class-assumptions.cpp` |
||||
|
||||
Solves the **circular dependency between type inference and call graph construction**: |
||||
|
||||
``` |
||||
$obj->method() needs $obj's type to bind method() |
||||
but type inference needs a complete call graph |
||||
→ Assumption breaks the cycle |
||||
``` |
||||
|
||||
Assumption sources: |
||||
- `@param ClassName $x` — parameter type |
||||
- `@return ClassName` — return type |
||||
- `@var ClassName` — local variable |
||||
- Constructor call `new ClassName()` → directly obtains the type |
||||
|
||||
Assumptions are made **before** type inference, used to bind the call graph. After type inference they are **validated** — an error is reported on mismatch. |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
For method call devirtualization: assumptions are available earlier than pure SSA analysis and can serve as the first stage of devirtualization (falling back when SSA is unavailable). |
||||
|
||||
--- |
||||
|
||||
### 2.8 Automatic Virtual Method Generation |
||||
|
||||
**File**: `compiler/pipes/generate-virtual-methods.cpp` |
||||
|
||||
When a method is overridden by subclasses, the base class method automatically becomes a dispatcher: |
||||
|
||||
```cpp |
||||
ReturnType f$Base$$method(instance_var, args...) { |
||||
if (instance_var.ce() == Child1::ce) |
||||
return f$Child1$$method(instance_cast<Child1>(instance_var), args...); |
||||
if (instance_var.ce() == Child2::ce) |
||||
return f$Child2$$method(instance_cast<Child2>(instance_var), args...); |
||||
// ... fallback to self |
||||
return f$Base$$method$$Base(instance_var, args...); |
||||
} |
||||
``` |
||||
|
||||
It also performs PHP 7.4+ type variance checks (parameter contravariance, return covariance). |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
The "runtime exact-type guard" in the current devirtualization plan is consistent with this idea. Its automatic generation of all dispatch branches + variance checking can be adopted. |
||||
|
||||
--- |
||||
|
||||
### 2.9 Performance Inspection Annotations |
||||
|
||||
**File**: `docs/kphp-language/best-practices/performance-inspections.md` |
||||
|
||||
Compile-time performance analysis, activated via annotations: |
||||
|
||||
```php |
||||
/** @kphp-warn-performance implicit-array-cast */ |
||||
function businessLogic() { ... } |
||||
``` |
||||
|
||||
Supported inspection items: |
||||
- `implicit-array-cast` — detect `array<int>` → `array<mixed>` implicit conversion (expensive copy) |
||||
- `array-merge-into` — detect merges that can be optimized via `array_merge_into` |
||||
- `array-reserve` — detect arrays that can be pre-sized |
||||
- `constant-execution-in-loop` — detect constant expressions inside loops |
||||
|
||||
Annotations propagate through the call chain to all reachable functions. |
||||
|
||||
**AOT adoption priority: P3** |
||||
|
||||
Similar to function coloring, this can serve as a compile-time static analysis plugin. `implicit-array-cast` has a huge performance impact on typed arrays and is worth detecting separately. |
||||
|
||||
--- |
||||
|
||||
### 2.10 Pooled Memory Allocator |
||||
|
||||
**File**: `runtime-common/core/memory-resource/unsynchronized_pool_resource.h` |
||||
|
||||
- Pre-allocates fixed-size buffers |
||||
- Small blocks (<16KB): slab allocation, graded by size (`free_chunks_[chunk_id]`), O(1) allocate/free |
||||
- Large blocks (≥16KB): red-black tree management (`huge_pieces_`), supporting defragmentation |
||||
- **Hard reset after each request** (`hard_reset()`), no per-object freeing needed |
||||
- Supports OOM handling memory reservation |
||||
|
||||
**AOT adoption priority: P4** |
||||
|
||||
Currently relies on Zend MM. For long-running CLI mode, a pool allocator can significantly reduce fragmentation and allocation overhead. But it requires replacing the entire memory management layer, a large effort. |
||||
|
||||
--- |
||||
|
||||
## 3. Toolchain Analysis |
||||
|
||||
### 3.1 Test Infrastructure |
||||
|
||||
KPHP has a three-layer test system: |
||||
|
||||
| Layer | Directory | Purpose | |
||||
|------|------|------| |
||||
| PHPT tests | `tests/phpt/` (75+ subdirectories) | PHP behavior compatibility tests | |
||||
| C++ unit tests | `tests/cpp/compiler/` `tests/cpp/runtime/` `tests/cpp/server/` | Compiler/runtime/server component tests | |
||||
| Python integration tests | `tests/python/tests/` | HTTP/RPC/multi-process integration tests | |
||||
|
||||
The test runner `tests/kphp_tester.py` supports: |
||||
- Tag mechanism (`@ok`, `@kphp_should_fail`, `@kphp_should_warn`, etc.) |
||||
- PHP version selection (`@php7.4`, `@php8`) |
||||
- Multi-process parallel execution (based on ThreadPool) |
||||
- TCP server management |
||||
- k2 mode (component compilation) compatibility |
||||
- Incremental compilation support (nocc distributed compilation) |
||||
|
||||
**AOT adoption**: |
||||
- Currently AOT only has two layers — `phpunit/` (PHPUnit) and `tests/compiler/` (PHPT) — lacking compiler internal unit tests and integration tests |
||||
- The tag mechanism is more flexible than pure PHPT — it can mark expected compile failures, expected warnings, etc. |
||||
- The Python test runner provides better CI integration capability |
||||
|
||||
### 3.2 Benchmark Framework |
||||
|
||||
**File**: `tests/benchmarks/` |
||||
|
||||
Uses the Go-written `ktest` tool for KPHP vs PHP performance comparison: |
||||
|
||||
``` |
||||
$ KPHP_ROOT=/path/to/repo/kphp ./ktest bench-vs-php tests/benchmarks/ |
||||
``` |
||||
|
||||
Benchmark coverage: |
||||
- `BenchmarkBasic.php` — basic operations |
||||
- `BenchmarkConcat.php` — string concatenation |
||||
- `BenchmarkExplode.php` — explode performance |
||||
- `BenchmarkMultiSwitch.php` — large switch |
||||
- `BenchmarkTmpString.php` — temporary string optimization effect |
||||
- `BenchmarkJson.php` / `BenchmarkFFI.php` — specific features |
||||
|
||||
**AOT adoption**: |
||||
- Can build a similar AOT vs PHP benchmark comparison suite |
||||
- Especially focus on scenarios the AOT compiler claims to optimize (such as typed property access, devirtualized calls) |
||||
|
||||
### 3.3 IDE Integration |
||||
|
||||
KPHP provides the **kphpstorm** IDE plugin (`docs/kphp-language/kphpstorm-ide-plugin/`), supporting: |
||||
- `@kphp-*` annotation syntax highlighting |
||||
- Type annotation completion |
||||
- Hints for KPHP-specific types |
||||
|
||||
**AOT adoption**: |
||||
- Currently the AOT compiler uses annotations like `@phpstan-*`; can consider providing VSCode/JetBrains plugins |
||||
|
||||
### 3.4 Incremental Compilation |
||||
|
||||
KPHP recompiles only changed files (based on CRC64 hashes): |
||||
|
||||
```cpp |
||||
// At the start of each generated file |
||||
//crc64 <content_hash> |
||||
//crc64_with_comments <hash_with_comments> |
||||
``` |
||||
|
||||
These hashes are compared against the previous generation results to determine which files need recompilation, including all upstream files that depend on them. |
||||
|
||||
**AOT adoption**: |
||||
- Currently the `build/` directory is fully regenerated; a similar incremental mechanism can be introduced to speed up iteration on large projects |
||||
|
||||
--- |
||||
|
||||
## 4. Syntax Compatibility Analysis |
||||
|
||||
### 4.1 PHP Versions Supported by KPHP |
||||
|
||||
KPHP targets the **PHP 7.4** language level, with some 8.0/8.1 features being added. |
||||
|
||||
### 4.2 Unsupported Features (Architectural Reasons) |
||||
|
||||
| Feature | Reason | |
||||
|------|------| |
||||
| Dynamic function/method calls (`call_user_func`) | Symbols cannot be resolved at compile time | |
||||
| `eval()` | Unknown at compile time | |
||||
| Dynamic class/function declarations | Symbol table must be complete at compile time | |
||||
| Reflection | Requires runtime metadata | |
||||
| Mock (PHPUnit) | Depends on Reflection + dynamic redefinition | |
||||
| Array internal pointers (`reset`/`current`/`next`) | Not consistent with reference semantics | |
||||
| PHP extension interop | Replaced by self-developed runtime | |
||||
|
||||
### 4.3 Unsupported Features (Not Implemented) |
||||
|
||||
| Feature | Status | |
||||
|------|------| |
||||
| Nested `list()` | Not implemented | |
||||
| Generators (`yield`) | Not implemented | |
||||
| Anonymous classes | Not implemented | |
||||
| Group use declarations | Not implemented | |
||||
| finally | Not implemented | |
||||
| `func_get_args` | Not implemented | |
||||
| References (except foreach by ref and reference parameters) | Partially supported | |
||||
| Interface appearing multiple times in a parent chain | Not supported | |
||||
| `insteadof` / trait renaming | Not supported | |
||||
|
||||
### 4.4 KPHP-Specific Annotations |
||||
|
||||
```php |
||||
// Function annotations |
||||
@kphp-inline // force inlining (GCC inline) |
||||
@kphp-flatten // aggressively inline all callees |
||||
@kphp-required // force compilation (for string callbacks) |
||||
@kphp-sync // forbid being resumable |
||||
@kphp-no-return // never returns (optimizes CFG) |
||||
@kphp-pure-function // pure function (callable on constant arrays) |
||||
@kphp-warn-unused-result // error on unused return value |
||||
@kphp-should-not-throw // forbid throwing exceptions |
||||
@kphp-throws {Class} // checked exceptions |
||||
@kphp-generic T1, T2 // generic functions |
||||
@kphp-color {color} // capability annotation |
||||
@kphp-warn-performance {...} // performance inspection |
||||
@kphp-disable-warnings {...} // suppress specific warnings |
||||
@kphp-profile // embed profiler |
||||
|
||||
// Class annotations |
||||
@kphp-serializable // serializable |
||||
@kphp-immutable-class // immutable class |
||||
@kphp-json {attr}={value} // JSON configuration |
||||
``` |
||||
|
||||
### 4.5 Syntax Differences from the AOT Compiler |
||||
|
||||
| Feature | KPHP | AOT Compiler | |
||||
|------|------|-------------| |
||||
| Base PHP version | 7.4 | 8.2+ | |
||||
| Enums | Not supported | Supported (PHP 8.1 enum) | |
||||
| Named arguments | Not supported | Supported | |
||||
| Match expressions | Not supported | Supported | |
||||
| Union types | Partial support | Supported | |
||||
| Nullsafe `?->` | Not supported | Supported | |
||||
| Constructor promotion | Not supported | Supported | |
||||
| `list()` destructuring | Partial | Full support | |
||||
| `break N` / `continue N` | Partial support | Supported | |
||||
| Typed arrays | Custom syntax `array<T>` | None (phpstan annotations) | |
||||
| Generic functions | `@kphp-generic` | None | |
||||
| Tuples / Shapes | Custom syntax | None | |
||||
| FFI | Supported (custom FFI) | None | |
||||
|
||||
### 4.6 Key Differences in the Type System |
||||
|
||||
KPHP's type system is much stricter than PHP's: |
||||
- **No type mixing allowed**: `f(42); f("string")` for the same `$arg` is a compile error |
||||
- **Typed arrays**: `array<int>` vs `array<string>` are different types; conversion requires explicit or implicit cast |
||||
- **mixed is expensive**: 16-byte tagged union + switch-case dispatch |
||||
- **Generic functions**: implemented via `@kphp-generic` compile-time specialization, similar to C++ templates |
||||
- **Variable splitting**: the same variable name may split into different names on different CFG paths (e.g. `$x` → `$x$v1`) |
||||
|
||||
--- |
||||
|
||||
## 5. Summary and Priority Recommendations |
||||
|
||||
| Priority | Technique | Difficulty | Benefit | Notes | |
||||
|--------|------|------|------|------| |
||||
| **P0** | Rewrite Rules DSL | Medium | High | Declarative optimization rules, extremely extensible | |
||||
| **P0** | Smart instanceof casts | Low | High | Directly improves object type narrowing + devirtualization | |
||||
| **P1** | Function specialization (multi-version) | Medium | High | More precise return types, eliminates mixed pollution | |
||||
| **P1** | Immutable constant markers | Low | Medium | Reduces COW; constant promotion foundation already exists | |
||||
| **P2** | Switch splitting | Medium | Medium | Multi-level break foundation already exists; optimization for specific scenarios | |
||||
| **P2** | Class Assumptions | High | High | Requires PHPDoc parsing infrastructure | |
||||
| **P2** | Automatic virtual method generation | Medium | High | Complements the devirtualization plan | |
||||
| **P3** | Performance inspection annotations | Low | Medium | Helps developers discover hidden performance issues | |
||||
| **P3** | Pipeline parallelism | High | Medium | Faster compilation of large projects | |
||||
| **P3** | Function coloring | Low | Low | Aids security/IO auditing | |
||||
| **P4** | Incremental compilation | Medium | Medium | Development experience optimization | |
||||
| **P5** | Resumable state machine | High | Medium | Requires actual async needs | |
||||
| **P5** | Pool allocator | Very high | High | Requires replacing the entire memory management | |
||||
@ -0,0 +1,525 @@ |
||||
# PeachPie Compiler Review: Roslyn Integration, Cross-Language Interop, and Type System |
||||
|
||||
> 2026-06-11 · Based on source review of /home/swoole/workspace/cpp/peachpie |
||||
|
||||
--- |
||||
|
||||
## 1. Architecture Overview |
||||
|
||||
PeachPie is a **PHP-to-.NET compiler based on Roslyn (Microsoft's .NET compiler platform)**, with roughly 710 C# source files and 272k lines of code. |
||||
|
||||
### Compilation Pipeline |
||||
|
||||
``` |
||||
PHP source |
||||
→ PhpSyntaxTree (Roslyn SyntaxTree for PHP — Syntax/) |
||||
→ SemanticModel + Symbols (Roslyn symbol system — Semantics/, Symbols/) |
||||
→ BoundControlFlowGraph (CFG with typed IR — Semantics/Graph/, FlowAnalysis/) |
||||
→ CIL Bytecode (EMIT — CodeGen/, Emitter/) |
||||
→ .NET Assembly (.dll / .exe) |
||||
``` |
||||
|
||||
| Stage | Component | Responsibility | |
||||
|------|------|------| |
||||
| Syntax parsing | `Syntax/` (PhpSyntaxTree, NodesFactory) | PHP → Roslyn SyntaxTree | |
||||
| Semantic binding | `Semantics/` (SemanticsBinder, BoundExpression) | Name resolution, method binding, type inference | |
||||
| Symbol system | `Symbols/` (SourceTypeSymbol, PEMethodSymbol...) | Roslyn symbol table for types/methods/properties | |
||||
| Dataflow analysis | `FlowAnalysis/` (FlowState, TypeRefMask, ExpressionAnalysis) | Type inference, unreachable code detection, conditional narrowing | |
||||
| CFG optimization | `FlowAnalysis/Passes/` (TransformationRewriter) | CFG rewriting, constantization, dead code elimination | |
||||
| Code generation | `CodeGen/` (CodeGenerator, GhostMethodBuilder) | IR → CIL instructions | |
||||
| Assembly output | `Emitter/` (PEModuleBuilder) | CIL → PE file (.dll/.exe) | |
||||
|
||||
### Core Files |
||||
|
||||
| File | Lines | Purpose | |
||||
|------|------|------| |
||||
| `CodeGen/Graph/BoundExpression.cs` | 5,742 | Core IR node definitions and CIL emission | |
||||
| `CodeGen/CodeGenerator.Emit.cs` | 4,396 | CIL instruction generation | |
||||
| `FlowAnalysis/ExpressionAnalysis.cs` | 2,911 | Expression-level type analysis | |
||||
| `Semantics/BoundExpression.cs` | 2,721 | Semantic binding expressions | |
||||
| `Runtime/Operators.cs` | 2,573 | Runtime implementation of PHP operators | |
||||
| `Runtime/PhpString.cs` | 2,047 | PHP string value type | |
||||
| `CodeGen/VariableReference.cs` | 1,863 | Variable references and address analysis | |
||||
| `Symbols/Source/SourceTypeSymbol.cs` | 1,795 | Source type symbols | |
||||
| `Runtime/Conversions.cs` | 1,603 | Type conversion runtime | |
||||
|
||||
### Comparison with AOT Compiler |
||||
|
||||
| Dimension | PeachPie | AOT Compiler | |
||||
|------|----------|-------------| |
||||
| Compilation target | PHP → CIL → .NET Assembly | PHP → C++ → binary | |
||||
| Compiler framework | Roslyn (C# compiler-as-a-library) | Self-built PHP AST → C++ string | |
||||
| IR form | BoundControlFlowGraph (Roslyn pattern) | PHP-Parser AST nodes | |
||||
| Type inference | FlowState + TypeRefMask bitset | SSA + manual type annotations | |
||||
| Symbol system | Roslyn Symbol hierarchy (complete) | Simplified ClassDef/FunctionDef | |
||||
| Output | .NET PE file (cross-platform) | Native binary (Linux/Mac/Windows) | |
||||
| Runtime | Peachpie.Runtime (PhpValue, PhpArray, PhpString) | phpx (C++ RAII wrapper around Zend API) | |
||||
| Cross-language interop | First-class citizen — PHP ⇄ C# bidirectional calls | FFI only (swoole_cc/cpp extension loading) | |
||||
| Parallel compilation | Parallel.ForEach function-level parallelism | Serial execution | |
||||
| Code size | ~272k lines of C# (including runtime) | ~15k lines of PHP | |
||||
| MSBuild integration | Complete SDK (`dotnet build`) | None | |
||||
|
||||
--- |
||||
|
||||
## 2. Reusable Innovative Designs |
||||
|
||||
### 2.1 Roslyn-Based Compiler Architecture |
||||
|
||||
**Files**: all of `Peachpie.CodeAnalysis/` |
||||
|
||||
PeachPie's most central design decision is to be **fully built on the Microsoft Roslyn compiler platform**. This means: |
||||
|
||||
- **Reuse Roslyn's symbol system**: `TypeSymbol`, `MethodSymbol`, `NamedTypeSymbol`, and other standard Roslyn types |
||||
- **Reuse Roslyn's metadata emission**: `PEModuleBuilder`, `PEAssemblyBuilder` directly generate PE files |
||||
- **Reuse Roslyn's diagnostics system**: `DiagnosticBag`, standardized Error/Warning mechanism |
||||
- **Native MSBuild integration**: PHP projects are standard .NET projects (`.csproj` style), compilable with `dotnet build` |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
Currently AOT built all compiler infrastructure from scratch. PeachPie's approach is a useful reference, but not suitable for direct porting (AOT's goal is to generate C++ rather than CIL). What can be adopted: |
||||
|
||||
- Decompose the compiler into standard stages — **Syntax → Semantic → IR → CodeGen** — with clear interfaces per stage |
||||
- Separate Preprocessor (symbol collection) into an independent Analyzer stage, similar to Roslyn's Compilation concept |
||||
- Define a unified `Diagnostic` type instead of scattered `fatalError()` / `SyntaxError` calls |
||||
|
||||
--- |
||||
|
||||
### 2.2 TypeRefMask: 64-bit Type Bitset |
||||
|
||||
**File**: `FlowAnalysis/TypeRef/TypeRefMask.cs` |
||||
|
||||
PeachPie uses `ulong` (64-bit) as the type mask: |
||||
|
||||
```csharp |
||||
public struct TypeRefMask { |
||||
ulong _mask; |
||||
// bits 0-61: type index (up to 62 different types) |
||||
// bit 62: IncludesSubclasses (type may include subclasses) |
||||
// bit 63: IsRef (value is a reference/alias) |
||||
} |
||||
``` |
||||
|
||||
Characteristics: |
||||
- **O(1) type comparison**: `(mask & type_bit) != 0` detects a type |
||||
- **Union types**: `mask1 | mask2` = contains both types |
||||
- **Type narrowing**: `mask & ~excluded_type_bit` = excludes a type |
||||
- **IsRef marker**: tracks whether a value is assigned by reference, affecting alias analysis |
||||
- **IncludesSubclasses**: distinguishes `exactly Class` vs `Class or subclass` |
||||
|
||||
Each `TypeRefContext` maintains a type registry mapping concrete .NET types (such as `System.Int64`, `Pchp.Core.PhpString`) to bit indices. |
||||
|
||||
**AOT adoption priority: P0** |
||||
|
||||
This is consistent with HHVM's trep idea — using a bitset to represent types. AOT's discrete constants like `TYPE_INT`, `TYPE_STRING` can be replaced with a bitset: |
||||
|
||||
```php |
||||
class TypeMask { |
||||
const BINT = 1 << 0; |
||||
const BFLOAT = 1 << 1; |
||||
const BSTRING = 1 << 2; |
||||
const BBOOL = 1 << 3; |
||||
const BARRAY = 1 << 4; |
||||
const BOBJECT = 1 << 5; |
||||
// Extension markers |
||||
const BEMPTY = 1 << 60; // empty value marker |
||||
const BSUBCLASS = 1 << 61; // allow subclasses |
||||
const BREFERENCE = 1 << 62; // reference marker |
||||
} |
||||
``` |
||||
|
||||
Advantage: `int|string` = `BINT|BSTRING`, type narrowing = `& ~excluded_bits`. |
||||
|
||||
--- |
||||
|
||||
### 2.3 FlowState + Worklist Dataflow Analysis |
||||
|
||||
**Files**: `FlowAnalysis/FlowState.cs`, `FlowAnalysis/Worklist.cs` |
||||
|
||||
PeachPie uses the classic **dataflow worklist algorithm** for type inference: |
||||
|
||||
```csharp |
||||
class FlowState { |
||||
TypeRefMask[] _varsType; // type mask for each variable |
||||
ulong _initializedMask; // whether variables are initialized |
||||
HashSet<NoteData> _notes; // additional info (such as function return points) |
||||
} |
||||
``` |
||||
|
||||
**Merge operation**: when two CFG paths converge, the `FlowState(state1, state2)` constructor computes: |
||||
- The **union** of type masks (all possible types) |
||||
- The **union** of initialization masks (initialized in either branch counts as initialized) |
||||
- The **intersection** of notes (only info present on both paths is retained) |
||||
|
||||
The Worklist processes blocks in topological order, re-enqueueing when state changes. |
||||
|
||||
**AOT adoption priority: P1** |
||||
|
||||
Currently AOT's SSA analysis does some type inference, but lacks: |
||||
- **Structured FlowState**: a unified variable type-state representation |
||||
- **Standard merge operation**: type merging at join points |
||||
- **Worklist iteration**: an iterative analysis framework reaching a fixed point |
||||
|
||||
--- |
||||
|
||||
### 2.4 ConditionBranch-Aware Type Narrowing |
||||
|
||||
**Files**: `FlowAnalysis/ConditionBranch.cs`, `FlowAnalysis/AnalysisFacts.cs` |
||||
|
||||
PeachPie's type analysis is **aware of the conditional branch of the current context**: |
||||
|
||||
```csharp |
||||
enum ConditionBranch { |
||||
AnyResult = 0, // ordinary evaluation |
||||
ToTrue = +1, // branch where the expression result is true |
||||
ToFalse = -1, // branch where the expression result is false |
||||
} |
||||
``` |
||||
|
||||
In conditional expressions, the analyzer carries the branch direction to propagate type info: |
||||
|
||||
```csharp |
||||
// if ($x instanceof MyClass) { ... } |
||||
// In the ToTrue branch: |
||||
// $x's type is narrowed, excluding types that cannot be MyClass |
||||
// In the ToFalse branch: |
||||
// $x's type is narrowed, excluding MyClass |
||||
|
||||
// if (is_int($x)) { ... } |
||||
// In the ToTrue branch: |
||||
// $x's type is narrowed to int |
||||
``` |
||||
|
||||
`AnalysisFacts.HandleSpecialFunctionCall()` registers type-check functions like `is_int`, `is_string`, `is_array`, `is_callable`, `function_exists`, `class_exists`, automatically narrowing variable types in branches. |
||||
|
||||
**AOT adoption priority: P1** |
||||
|
||||
Currently `SsaTypeOptimizer` does some instanceof narrowing, but: |
||||
- Does not support narrowing via built-in type-check functions like `is_int()` / `is_string()` |
||||
- Does not support constant folding of existence checks like `class_exists()` / `function_exists()` |
||||
- Can directly adopt `AnalysisFacts`'s "known type-check function registry" pattern |
||||
|
||||
--- |
||||
|
||||
### 2.5 PhpValue Tagged Union Design |
||||
|
||||
**File**: `Peachpie.Runtime/PhpValue.cs` |
||||
|
||||
PeachPie's runtime value type uses an ingenious C# tagged union: |
||||
|
||||
```csharp |
||||
[StructLayout(LayoutKind.Sequential)] |
||||
public readonly partial struct PhpValue { |
||||
readonly PhpTypeCode _type; // 1-byte type tag |
||||
|
||||
// Explicit-layout union: two fields occupy the same memory |
||||
[StructLayout(LayoutKind.Explicit)] |
||||
struct ValueField { |
||||
[FieldOffset(0)] public bool @bool; |
||||
[FieldOffset(0)] public long @long; |
||||
[FieldOffset(0)] public double @double; |
||||
} |
||||
|
||||
[StructLayout(LayoutKind.Explicit)] |
||||
struct ObjectField { |
||||
[FieldOffset(0)] public object @object; |
||||
[FieldOffset(0)] public string @string; |
||||
[FieldOffset(0)] public PhpString.Blob blob; |
||||
[FieldOffset(0)] public PhpArray array; |
||||
[FieldOffset(0)] public PhpAlias alias; |
||||
} |
||||
|
||||
readonly ValueField _value; // value-type storage |
||||
readonly ObjectField _obj; // reference-type storage |
||||
} |
||||
``` |
||||
|
||||
Memory layout: `PhpValue` = `PhpTypeCode` (1 byte) + padding + `ValueField` (8 bytes) + `ObjectField` (8 bytes, pointer) ≈ 24 bytes. |
||||
|
||||
Advantages of this design: |
||||
- **readonly struct** — no GC overhead, allocatable on the stack |
||||
- **Explicit union** — value types and reference types share space, compact |
||||
- **PhpAlias mechanism** — abstracts PHP references (copy-on-write) via `PhpAlias` rather than directly copying values |
||||
- **MutableString** — distinguishes immutable string from writable MutableString (for string concatenation optimization) |
||||
|
||||
**AOT adoption priority: P3** |
||||
|
||||
Currently AOT uses `Variant` (based on Zend `zval`) as the dynamic type. What can be adopted: |
||||
- PhpString's MutableString separation (string builder pattern) |
||||
- PhpAlias's reference semantics abstraction |
||||
- But replacing `Variant` entirely is a huge effort, so low priority |
||||
|
||||
--- |
||||
|
||||
### 2.6 GhostMethodBuilder: PHP Method ⇄ C# Method Adaptation |
||||
|
||||
**File**: `CodeGen/GhostMethodBuilder.cs` |
||||
|
||||
PeachPie's most distinctive feature is **automatically generating ghost stub methods** so PHP methods can be called directly from C#: |
||||
|
||||
```csharp |
||||
// Generate C#-callable wrappers for PHP methods: |
||||
// - Handle parameter type conversion (PhpValue → CLR type) |
||||
// - Handle return type conversion (CLR type → PhpValue) |
||||
// - Build and pass PhpContext |
||||
// - Support explicit interface override |
||||
static MethodSymbol CreateGhostOverload( |
||||
MethodSymbol original, NamedTypeSymbol containingtype, |
||||
PEModuleBuilder module, DiagnosticBag diagnostic, |
||||
TypeSymbol ghostreturn, ImmutableArray<ParameterSymbol> ghostparams, |
||||
bool phphidden = false, MethodSymbol explicitOverride = null) |
||||
``` |
||||
|
||||
Ghost methods enable: |
||||
- Type-safe interfaces when C# calls PHP methods |
||||
- PHP implementing C# interfaces (IMethod, INotifyPropertyChanged, etc.) |
||||
- PHP classes usable as .NET generic parameters |
||||
|
||||
**AOT adoption priority: P4** |
||||
|
||||
Currently AOT does not support PHP calling C++ (or vice versa). If a bidirectional interop layer is needed in the future, the ghost stub pattern is worth referencing. |
||||
|
||||
--- |
||||
|
||||
### 2.7 DelayedTransformations: Parallel-Safe Deferred Transformations |
||||
|
||||
**File**: `FlowAnalysis/Passes/DelayedTransformations.cs` |
||||
|
||||
During the parallel analysis phase, certain transformations (such as marking unreachable functions, promoting conditional functions to unconditional ones) cannot directly modify shared state. PeachPie uses the **deferred transformation** pattern: |
||||
|
||||
```csharp |
||||
class DelayedTransformations { |
||||
ConcurrentBag<SourceRoutineSymbol> UnreachableRoutines; |
||||
ConcurrentBag<SourceTypeSymbol> UnreachableTypes; |
||||
ConcurrentBag<SourceFunctionSymbol> FunctionsMarkedAsUnconditional; |
||||
// Collected thread-safely during parallel analysis |
||||
// Applied serially via Apply() after analysis completes |
||||
} |
||||
``` |
||||
|
||||
Analysis threads only place the objects to be transformed into the `ConcurrentBag`, and after analysis completes a single thread calls `Apply()`. |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
AOT is currently serial and doesn't need this. But if parallel compilation is introduced in the future (see KPHP review 2.3), deferred transformation is the foundational pattern for thread safety. |
||||
|
||||
--- |
||||
|
||||
### 2.8 Native MSBuild Integration (Peachpie.NET.Sdk) |
||||
|
||||
**File**: `Peachpie.NET.Sdk/` |
||||
|
||||
PeachPie is not just a compiler — it is a **complete .NET SDK**: |
||||
|
||||
``` |
||||
dotnet new classlibrary -o MyPhpLib # create a PHP class library project |
||||
dotnet build # compile PHP → .NET DLL |
||||
dotnet run # run the compiled program |
||||
dotnet publish # publish as a self-contained app |
||||
``` |
||||
|
||||
Implemented via MSBuild targets/props: |
||||
- `build/peachpie.targets` — defines the compilation task |
||||
- `Peachpie.NET.Sdk.nuspec` — NuGet package definition |
||||
- `BuildTask.cs` — MSBuild compilation task |
||||
|
||||
This means PHP projects can seamlessly use the .NET ecosystem: NuGet package references, project references, conditional compilation, multi-targeting, etc. |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
AOT currently uses `php bin/tpc.php <project>` on the command line. What can be adopted: |
||||
- Create a Composer plugin or CLI phar package for the AOT compiler |
||||
- Define a JSON Schema for `project.yml` (similar to `.csproj`) |
||||
- Support a unified entry point like `composer build` or `php-aot build` |
||||
|
||||
--- |
||||
|
||||
### 2.9 Lazily Evaluable AnalysisFacts |
||||
|
||||
**File**: `FlowAnalysis/AnalysisFacts.cs` |
||||
|
||||
PeachPie performs constant evaluation of a large number of PHP runtime functions at compile time: |
||||
|
||||
| Function | Evaluation strategy | |
||||
|------|---------| |
||||
| `function_exists(X)` | Check if symbol X exists in the PE assembly → fold to `true` | |
||||
| `class_exists(X)` | Check if type X exists in the PE assembly → fold to `true`/`false` | |
||||
| `method_exists(X, M)` | Check if method M exists in type X → fold to `true`/`false` | |
||||
| `defined(CONST)` | Check if the constant exists → fold to `true`/`false` | |
||||
| `is_callable(F)` | Check if F is unconditionally declared → fold to `true` | |
||||
| `dirname(__FILE__)` | Compile-time path computation → `__DIR__` | |
||||
| `basename(__FILE__)` | Compile-time filename extraction → string constant | |
||||
|
||||
These evaluations leverage PeachPie's "PE assembly" concept — already-compiled .NET assemblies contain complete type/method/constant metadata that can be **queried at compile time**. |
||||
|
||||
**AOT adoption priority: P1** |
||||
|
||||
Currently `FuncCallOptimizer` only does the most basic constant folding (`strlen("abc")` → `3`). It can be extended to compile-time evaluation of reflective functions like `function_exists`, `class_exists`, `defined` — provided a complete symbol table is built in Preprocessor. |
||||
|
||||
--- |
||||
|
||||
### 2.10 Conditional Declaration Detection and Unreachable Code Elimination |
||||
|
||||
**Files**: `FlowAnalysis/Passes/DelayedTransformations.cs`, `FlowAnalysis/Passes/TransformationRewriter.cs` |
||||
|
||||
PeachPie can detect **conditional declarations** (`if (condition) { function foo() {} }`) and optimize them: |
||||
|
||||
- If analysis proves the condition is always true → the function is marked as unconditionally declared |
||||
- If analysis proves the condition is always false → the function/class is marked as Unreachable and not compiled |
||||
|
||||
This enables writing C-like conditional compilation patterns in PHP: |
||||
|
||||
```php |
||||
if (PHP_VERSION_ID >= 80000) { |
||||
function newFeature() { ... } // not compiled on lower versions |
||||
} |
||||
``` |
||||
|
||||
**AOT adoption priority: P2** |
||||
|
||||
Currently AOT compiles every scanned function regardless of reachability. For projects with multi-version PHP compatibility code, conditional declaration elimination can reduce compilation artifact size. |
||||
|
||||
--- |
||||
|
||||
## 3. Toolchain Analysis |
||||
|
||||
### 3.1 Test Infrastructure |
||||
|
||||
PeachPie has 529 test files (PHP files) distributed across functional directories: |
||||
|
||||
| Directory | Content | |
||||
|------|------| |
||||
| `tests/arrays/` | Array operation tests (including a `lazy_copy` subdirectory) | |
||||
| `tests/classes/` | Class/object tests | |
||||
| `tests/functions/` | Function call tests | |
||||
| `tests/generators/` | Generator/yield tests | |
||||
| `tests/strings/` | String operation tests | |
||||
| `tests/operators/` | Operator tests | |
||||
| `tests/transformations/` | Compiler transformation/optimization tests | |
||||
| `tests/constants/` | Constant tests | |
||||
| `tests/constructs/` | Language construct tests | |
||||
| `tests/traits/` | Trait tests | |
||||
| `tests/reflection/` | Reflection tests | |
||||
| `tests/spl/` | SPL tests | |
||||
| `tests/bcmath/` `tests/hash/` `tests/pcre/` | Extension tests | |
||||
| `tests/pdo/` `tests/ftp/` `tests/openssl/` | Database/network extensions | |
||||
| `tests/gd/` `tests/xml/` `tests/zip/` | Graphics/XML/ZIP extensions | |
||||
| `tests/web/` `tests/scripting/` | Web/scripting integration tests | |
||||
|
||||
Test execution: compiled assemblies are executed through the .NET test framework (xUnit/NUnit). |
||||
|
||||
**AOT adoption:** |
||||
- The `tests/transformations/` directory specifically tests compiler optimizations — AOT can build similar optimization-correctness tests |
||||
- `tests/arrays/lazy_copy/` specifically tests copy-on-write behavior — AOT can also build dedicated COW tests |
||||
|
||||
### 3.2 Deep Visual Studio Integration |
||||
|
||||
PeachPie provides a complete IDE experience: |
||||
- **Visual Studio Extension** — project management, IntelliSense, debugging, performance analysis |
||||
- **VS Code / Rider support** — via OmniSharp / LSP |
||||
- **NuGet package management** — PHP libraries can be published and referenced as NuGet packages |
||||
|
||||
**AOT adoption:** |
||||
- Can provide a VSCode extension (Task integration + project templates) |
||||
- AOT's current `project.yml` can use a JSON Schema to provide IDE auto-completion |
||||
|
||||
### 3.3 Command-Line Toolchain |
||||
|
||||
```bash |
||||
# PeachPie's CLI experience |
||||
dotnet peach build # compile a PHP project |
||||
dotnet peach run # run the compiled program |
||||
dotnet peach publish # self-contained publishing |
||||
dotnet peach add <package> # add a dependency |
||||
``` |
||||
|
||||
**AOT adoption:** |
||||
```bash |
||||
# A similar CLI could be designed |
||||
php-aot build # compile a project |
||||
php-aot run # compile and run |
||||
php-aot new <name> # create a new project |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 4. Type System and Interop Analysis |
||||
|
||||
### 4.1 PeachPie Type Mapping |
||||
|
||||
| PHP type | PeachPie runtime type | .NET CLR type | |
||||
|----------|-------------------|---------------| |
||||
| null | `PhpTypeCode.Null` | `null` (any reference type) | |
||||
| bool | `PhpTypeCode.Boolean` | `bool` | |
||||
| int | `PhpTypeCode.Long` | `long` | |
||||
| float | `PhpTypeCode.Double` | `double` | |
||||
| string | `PhpTypeCode.String` / `MutableString` | `string` / `PhpString` | |
||||
| array | `PhpTypeCode.PhpArray` | `PhpArray` | |
||||
| object | `PhpTypeCode.Object` | `object` (concrete class) | |
||||
| reference | `PhpTypeCode.Alias` | `PhpAlias` | |
||||
|
||||
### 4.2 PHP ⇄ C# Interop |
||||
|
||||
PeachPie's bidirectional interop is its most prominent differentiator: |
||||
|
||||
**C# calling PHP:** |
||||
```csharp |
||||
// Compiled PHP classes become .NET classes; C# can directly new and call them |
||||
var phpObj = new MyPhpClass(ctx); |
||||
phpObj.someMethod(arg1, arg2); |
||||
``` |
||||
|
||||
**PHP calling C#:** |
||||
```php |
||||
// .NET types can be used directly in PHP |
||||
$list = new \System\Collections\Generic\List<int>; |
||||
$list->Add(42); |
||||
``` |
||||
|
||||
Interop implementation relies on: |
||||
- `GhostMethodBuilder` generating adapter methods |
||||
- `ConversionsExtensions` handling automatic PhpValue ↔ CLR type conversion |
||||
- `DynamicOperationFactory` handling dynamic method call forwarding |
||||
|
||||
### 4.3 Syntax Differences from the AOT Compiler |
||||
|
||||
| Feature | PeachPie | AOT Compiler | |
||||
|------|----------|-------------| |
||||
| Base PHP version | 8.0+ (target) | 8.2+ | |
||||
| Type annotations | Optional (gradually enriched) | Optional (phpstan annotations) | |
||||
| Namespaces | Standard PHP | Standard PHP | |
||||
| Generics | None | None | |
||||
| C# interop | Complete (first-class citizen) | FFI extension only | |
||||
| .NET ecosystem | Fully compatible | Not relevant | |
||||
| MSBuild integration | Complete | None | |
||||
| Reflection | Partial support | Not supported | |
||||
| yield/generator | Supported | Supported | |
||||
|
||||
--- |
||||
|
||||
## 5. Summary and Priority Recommendations |
||||
|
||||
| Priority | Technique | Difficulty | Benefit | Notes | |
||||
|--------|------|------|------|------| |
||||
| **P0** | TypeRefMask bitset type system | Medium | Very high | Union types, type narrowing, non-null markers — foundation for all optimization passes | |
||||
| **P1** | ConditionBranch type narrowing | Low | High | Automatic narrowing via `is_int()`/`is_string()` check functions, not covered by current SSA | |
||||
| **P1** | AnalysisFacts compile-time evaluation | Medium | High | Compile-time folding of `function_exists`/`class_exists`/`defined` | |
||||
| **P1** | FlowState worklist analysis framework | Medium | High | Structured variable type state, standard merge operation | |
||||
| **P2** | Roslyn-style compiler layering | High | Medium | Clear Syntax→Semantic→IR→CodeGen layering; current Preprocessor/CompilerBase responsibilities are mixed | |
||||
| **P2** | Conditional declaration detection | Low | Medium | Eliminate unreachable functions/classes, reduce compilation artifact size | |
||||
| **P2** | MSBuild integration / CLI unification | Low | Medium | Standardized project configuration, CI-friendly | |
||||
| **P3** | PhpValue tagged union | High | Medium | Compact memory layout, but replacing Variant/zval is a large effort | |
||||
| **P3** | DelayedTransformations | Low | Low | Only meaningful under parallel compilation | |
||||
| **P4** | GhostMethodBuilder interop | Very high | Low | Requires .NET or a similar FFI runtime | |
||||
| **P4** | Deep IDE integration | High | Low | Limited ROI for a VSCode extension | |
||||
|
||||
### Key Takeaways |
||||
|
||||
1. **PeachPie's biggest advantage is .NET ecosystem integration** — this is a natural advantage from its architectural choice rather than any single technical innovation. AOT can adopt its "compiler SDK + standardized CLI" thinking without porting specific techniques. |
||||
|
||||
2. **TypeRefMask bitset is the most directly portable design** — representing types with a 64-bit bitset while supporting union types, subclass markers, and reference markers. This is a common feature of HHBBC's trep and KPHP's type analysis, indicating that the bitset type lattice is the best practice for PHP compilers. |
||||
|
||||
3. **The ConditionBranch pattern is a low-cost, high-benefit enhancement** — carrying "expected result" info in conditional branch analysis enables type-check functions to automatically narrow variable types. AOT's SSA analysis can adopt it immediately. |
||||
|
||||
4. **GhostMethodBuilder reveals a general interop pattern** — generating adapter/thunk methods to bridge the calling conventions of two languages. Although AOT targets C++ rather than .NET, if cross-language call needs arise (such as PHP calling C extensions), this pattern can be reused. |
||||
|
||||
5. **PeachPie's roughly 530 tests** are notably fewer than KPHP (75+ directories) and HHVM (14,675), indicating relatively lower maturity. But its way of categorizing tests by functional module and optimization type is worth referencing. |
||||
@ -0,0 +1,682 @@ |
||||
# PHP Zend Optimizer / OPcache / JIT Design Analysis: Mechanisms That Can Be Introduced into an AOT Compiler |
||||
|
||||
This document analyzes the implementation of the Zend Optimizer, OPcache, and JIT in php-src (v8.4.14), identifying the design patterns, algorithms, and data-flow frameworks that can be introduced into an AOT compiler. |
||||
|
||||
Source locations: |
||||
- Optimizer: `~/soft/php/php-8.4.14/Zend/Optimizer/` |
||||
- OPcache: `~/soft/php/php-8.4.14/ext/opcache/` |
||||
- JIT: `~/soft/php/php-8.4.14/ext/opcache/jit/` |
||||
|
||||
--- |
||||
|
||||
## Overview |
||||
|
||||
PHP's optimizer is one of the most mature high-end optimizers for dynamic languages in the field. It contains complete compiler optimization infrastructure, including full SSA (Static Single Assignment) construction, e-SSA (Extended SSA) type/range inference, SCCP (Sparse Conditional Constant Propagation), escape analysis, dead code elimination, call graph analysis, profile-guided tracing JIT, and more. |
||||
|
||||
An AOT compiler can borrow from the following aspects: |
||||
|
||||
| Priority | Design/Module | Implementation Scale | Benefit | |
||||
|--------|----------|---------|------| |
||||
| P1 | Pass Pipeline architecture | ~100 lines of framework | Clear architecture, pluggable, split into O0/O1/O2 | |
||||
| P1 | SSA + e-SSA construction | Medium | Foundation for all advanced optimizations | |
||||
| P1 | Type inference (Type & Range Inference) | Medium | Precise type deduction capability | |
||||
| P2 | SCDF generic data-flow framework | ~300 lines | Reusable for SCCP/type inference/optimization | |
||||
| P2 | SCCP constant propagation | Medium | Conditional constant folding + unreachable code elimination | |
||||
| P2 | DCE dead code elimination | ~400 lines | Worklist-driven precise DCE | |
||||
| P3 | Escape Analysis | ~500 lines | Stack allocation, reference count elimination | |
||||
| P3 | Call Graph | ~400 lines | Cross-function analysis, inlining decisions, dead code | |
||||
| P4 | JIT IR framework | Very heavy | Reference the abstraction levels of its IR design | |
||||
| P4 | OPcache persistence/File Cache | Light | Serialized IR cache after optimization | |
||||
|
||||
--- |
||||
|
||||
## 1. Pass Pipeline Architecture |
||||
|
||||
### Design |
||||
|
||||
PHP defines 16 optimization passes, each corresponding to a bitmask bit (`zend_optimizer.h:28-46`): |
||||
|
||||
```c |
||||
#define ZEND_OPTIMIZER_PASS_1 (1<<0) // Simple local optimization (constant replacement/folding) |
||||
#define ZEND_OPTIMIZER_PASS_2 (1<<1) // |
||||
#define ZEND_OPTIMIZER_PASS_3 (1<<2) // Jump optimization |
||||
#define ZEND_OPTIMIZER_PASS_4 (1<<3) // INIT_FCALL_BY_NAME -> DO_FCALL |
||||
#define ZEND_OPTIMIZER_PASS_5 (1<<4) // CFG optimization (block pass) |
||||
#define ZEND_OPTIMIZER_PASS_6 (1<<5) // DFA optimization (type/range inference → single function) |
||||
#define ZEND_OPTIMIZER_PASS_7 (1<<6) // CALL GRAPH optimization (cross-function analysis) |
||||
#define ZEND_OPTIMIZER_PASS_8 (1<<7) // SCCP (constant propagation) |
||||
#define ZEND_OPTIMIZER_PASS_9 (1<<8) // Temporary variable optimization |
||||
#define ZEND_OPTIMIZER_PASS_10 (1<<9) // NOP removal |
||||
#define ZEND_OPTIMIZER_PASS_11 (1<<10) // Merge identical constants |
||||
#define ZEND_OPTIMIZER_PASS_12 (1<<11) // Adjust stack usage |
||||
#define ZEND_OPTIMIZER_PASS_13 (1<<12) // Remove unused variables |
||||
#define ZEND_OPTIMIZER_PASS_14 (1<<13) // DCE (dead code elimination) |
||||
#define ZEND_OPTIMIZER_PASS_15 (1<<14) // Collect constants (unsafe) |
||||
#define ZEND_OPTIMIZER_PASS_16 (1<<15) // Function inlining |
||||
``` |
||||
|
||||
### Pipeline Scheduling |
||||
|
||||
The `zend_optimize()` function (`zend_optimizer.c:1067-1183`) executes each pass in order, with each pass only processing the results produced by already-executed passes: |
||||
|
||||
``` |
||||
pass1 (constant folding) → pass3 (jump optimization) → pass4 (function call optimization) |
||||
→ pass5 (CFG) → pass6 (DFA + type inference) → pass9 (temporary variables) |
||||
→ pass10 (NOP removal) → pass11 (constant merging) → pass13 (variable cleanup) → ... |
||||
``` |
||||
|
||||
When `PASS_6 + PASS_7` are enabled simultaneously, `zend_optimize_script()` takes a more complex call graph path: |
||||
``` |
||||
build_call_graph → zend_optimize (per-func) → analyze_call_graph |
||||
→ build call_map → dfa_analyze_op_array (per-func) |
||||
→ dfa_optimize_op_array (per-func, with call context) |
||||
→ pass9 → pass11 → pass13 → pass12 (stack adjust) → redo_pass_two |
||||
``` |
||||
|
||||
### Key Design Points |
||||
|
||||
**1. Bitmask switches + registered passes** |
||||
|
||||
Users can combine arbitrary passes by bitmask. It also supports `zend_optimizer_register_pass()` to register external passes (such as JIT's own optimization passes): |
||||
|
||||
```c |
||||
static struct { |
||||
zend_optimizer_pass_t pass[ZEND_OPTIMIZER_MAX_REGISTERED_PASSES]; |
||||
int last; |
||||
} zend_optimizer_registered_passes; |
||||
``` |
||||
|
||||
Registered passes execute after all built-in passes (`zend_optimizer_call_registered_passes`). |
||||
|
||||
**2. Optional dump output per pass** |
||||
|
||||
Controlled via `debug_level`, supporting output of the intermediate representation before/after any pass for debugging and performance analysis. |
||||
|
||||
**3. Dual-layer optimization: per-function and script-level** |
||||
|
||||
- `zend_optimize(op_array, ctx)` — single-function optimization, conservative (does not use cross-function information) |
||||
- `zend_optimize_script(script, ...)` — whole-script optimization with call graph, enabling cross-function optimization |
||||
|
||||
### AOT Takeaways |
||||
|
||||
The AOT compiler can define a similar pass pipeline: |
||||
|
||||
```php |
||||
enum AotPass: int { |
||||
case CONSTANT_FOLD = 1 << 0; |
||||
case TYPE_CHECK_INSERT = 1 << 1; |
||||
case ESCAPE_ANALYSIS = 1 << 2; |
||||
case DEVIRTUALIZE = 1 << 3; |
||||
case DEAD_CODE_ELIM = 1 << 4; |
||||
case FUNCTION_INLINE = 1 << 5; |
||||
case LOOP_OPTIMIZE = 1 << 6; |
||||
case BOX_ALLOC_ELIM = 1 << 7; |
||||
} |
||||
``` |
||||
|
||||
Combined by optimization level: |
||||
|
||||
```php |
||||
const O0 = AotPass::TYPE_CHECK_INSERT->value; // Required baseline code generation |
||||
const O1 = O0 | AotPass::CONSTANT_FOLD->value; // Basic optimization |
||||
const O2 = O1 | AotPass::DEVIRTUALIZE->value | AotPass::FUNCTION_INLINE->value; |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 2. SSA (Static Single Assignment) + e-SSA |
||||
|
||||
### Data Structures |
||||
|
||||
SSA is built on top of the control flow graph (CFG): |
||||
|
||||
**CFG (`zend_cfg.h:84-92`):** |
||||
```c |
||||
typedef struct _zend_cfg { |
||||
int blocks_count; // Number of basic blocks |
||||
int edges_count; // Number of edges |
||||
zend_basic_block *blocks; // Array of basic blocks |
||||
int *predecessors; // Predecessor list |
||||
uint32_t *map; // opnum → block mapping |
||||
uint32_t flags; |
||||
} zend_cfg; |
||||
|
||||
typedef struct _zend_basic_block { |
||||
int *successors; // Successor block indices |
||||
uint32_t flags; |
||||
uint32_t start; // Starting opcode |
||||
uint32_t len; // Number of opcodes |
||||
int successors_count; |
||||
int predecessors_count; |
||||
int idom; // Immediate dominator |
||||
int loop_header; // Nearest loop header |
||||
int level; // Dominator tree depth |
||||
int children; // Linked list of dominated child blocks |
||||
} zend_basic_block; |
||||
``` |
||||
|
||||
**SSA (`zend_ssa.h:135-143`):** |
||||
```c |
||||
typedef struct _zend_ssa { |
||||
zend_cfg cfg; // Control flow graph |
||||
int vars_count; // Number of SSA variables |
||||
int sccs; // Number of strongly connected components |
||||
zend_ssa_block *blocks; // φ functions per basic block |
||||
zend_ssa_op *ops; // use-def information per instruction |
||||
zend_ssa_var *vars; // def-use chain per SSA variable |
||||
zend_ssa_var_info *var_info; // Type inference results (type bitmask + range) |
||||
} zend_ssa; |
||||
``` |
||||
|
||||
**SSA Op (`zend_ssa.h:82-92`):** |
||||
```c |
||||
typedef struct _zend_ssa_op { |
||||
int op1_use; |
||||
int op2_use; |
||||
int result_use; |
||||
int op1_def; // SSA variable defined by this instruction |
||||
int op2_def; |
||||
int result_def; |
||||
int op1_use_chain; // use-def chain |
||||
int op2_use_chain; |
||||
int res_use_chain; |
||||
} zend_ssa_op; |
||||
``` |
||||
|
||||
### e-SSA: Extended SSA with Pi Nodes |
||||
|
||||
This is one of the most elegant designs of the PHP optimizer. A Pi node is a special kind of φ function used to represent type/range constraints inferred from conditional branches. |
||||
|
||||
**Pi constraints (`zend_ssa.h:42-59`):** |
||||
```c |
||||
typedef struct _zend_ssa_range_constraint { |
||||
zend_ssa_range range; // Range constraint [min, max] |
||||
int min_var; // Symbolic lower-bound variable |
||||
int max_var; // Symbolic upper-bound variable |
||||
zend_ssa_negative_lat negative; // Negation potential |
||||
} zend_ssa_range_constraint; |
||||
|
||||
typedef struct _zend_ssa_type_constraint { |
||||
uint32_t type_mask; // Type mask (narrowed type after AND operation) |
||||
zend_class_entry *ce; // Class entry (for instanceof) |
||||
} zend_ssa_type_constraint; |
||||
|
||||
typedef union _zend_ssa_pi_constraint { |
||||
zend_ssa_range_constraint range; |
||||
zend_ssa_type_constraint type; |
||||
} zend_ssa_pi_constraint; |
||||
``` |
||||
|
||||
**How it works:** For a condition like `if ($x > 0)`: |
||||
- In the truthy branch, insert `Pi($x, range[1, LONG_MAX])` — constrain `$x`'s SSA variable to the range > 0 |
||||
- In the falsy branch, insert `Pi($x, range[LONG_MIN, 0])` — constrain `$x` to ≤ 0 |
||||
|
||||
This allows subsequent optimizations within a branch to use refined type/range information without altering the explicit assignment chain of the original variable. |
||||
|
||||
### SSA Construction Flow |
||||
|
||||
``` |
||||
1. zend_build_cfg() → Build the control flow graph (including dominator tree, loop detection) |
||||
2. zend_build_dfg() → Build the data flow graph (compute use/def sets) |
||||
3. zend_build_ssa() → Place φ functions → rename variables → build SSA form |
||||
4. zend_ssa_compute_use_def_chains() → Connect use-def chains |
||||
5. zend_ssa_find_sccs() → Find strongly connected components (for type inference) |
||||
6. zend_ssa_inference() → Type inference + range inference (populate var_info) |
||||
``` |
||||
|
||||
### AOT Takeaways |
||||
|
||||
The AOT compiler does not need SSA form (because it generates C++ code, not direct register manipulation), but the following e-SSA concepts can be used directly: |
||||
|
||||
1. **The Pi constraint concept:** Insert type-narrowing markers in conditional branches so that variables inside a branch have more precise types. This directly corresponds to the implementation basis of the TypeSpecifier / Type Narrowing (#7) mentioned earlier. |
||||
|
||||
2. **Type & Range information associated with every expression:** Similar to the SSA `var_info` design, AOT can maintain a `{type_mask, range, ce}` triple for every variable/expression in the FunctionContext. |
||||
|
||||
3. **Use-def chains for optimization decisions:** When determining whether a variable has exactly one `use`, SSA's use_chain provides O(1) lookup. |
||||
|
||||
--- |
||||
|
||||
## 3. Type Inference (Type & Range Inference) |
||||
|
||||
### The Type System |
||||
|
||||
PHP uses a bitmask to represent type information (defined in `zend_type_info.h`), which is its most distinctive design: |
||||
|
||||
```c |
||||
#define MAY_BE_UNDEF (1<< 0) |
||||
#define MAY_BE_NULL (1<< 1) |
||||
#define MAY_BE_FALSE (1<< 2) |
||||
#define MAY_BE_TRUE (1<< 3) |
||||
#define MAY_BE_LONG (1<< 4) |
||||
#define MAY_BE_DOUBLE (1<< 5) |
||||
#define MAY_BE_STRING (1<< 6) |
||||
#define MAY_BE_ARRAY (1<< 7) |
||||
#define MAY_BE_OBJECT (1<< 8) |
||||
#define MAY_BE_RESOURCE (1<< 9) |
||||
#define MAY_BE_REFERENCE (1<<10) |
||||
#define MAY_BE_CALLABLE (1<<11) |
||||
#define MAY_BE_ITERABLE (1<<12) |
||||
#define MAY_BE_VOID (1<<13) |
||||
#define MAY_BE_INDIRECT (1<<14) |
||||
|
||||
// Convenient combinations |
||||
#define MAY_BE_ANY (MAY_BE_NULL|MAY_BE_FALSE|MAY_BE_TRUE|...) |
||||
#define MAY_BE_TRUTHY (MAY_BE_TRUE|MAY_BE_LONG|... /* not 0/''/[]/null */) |
||||
#define MAY_BE_FALSEY (MAY_BE_UNDEF|MAY_BE_NULL|MAY_BE_FALSE|...) |
||||
``` |
||||
|
||||
**Core advantage:** Bit operations are extremely fast. Type operations (merge/intersection/difference) require only a single AND/OR/NOT instruction: |
||||
|
||||
```c |
||||
// Merge the types of two variables |
||||
uint32_t result_type = info1 | info2; |
||||
|
||||
// Check whether it might be a string |
||||
if (info & MAY_BE_STRING) { ... } |
||||
|
||||
// Intersection |
||||
uint32_t common = info1 & info2; |
||||
``` |
||||
|
||||
### Range Inference |
||||
|
||||
Each SSA variable carries a `zend_ssa_range { min, max, underflow, overflow }`: |
||||
|
||||
```c |
||||
typedef struct _zend_ssa_range { |
||||
zend_long min; |
||||
zend_long max; |
||||
bool underflow; // Whether there is an underflow risk |
||||
bool overflow; // Whether there is an overflow risk |
||||
} zend_ssa_range; |
||||
``` |
||||
|
||||
The core algorithm (`zend_inference.c:1071`) is based on V. Campos's "Speed and Precision in Range Analysis, SBLP'12" paper: |
||||
|
||||
1. **Warmup phase (16 passes):** Propagate ranges over SCCs (strongly connected components), using widening to accelerate convergence |
||||
2. **Narrowing phase:** Gradually narrow the ranges, eliminating the over-approximation caused by widening |
||||
3. **Zend Engine-specific arithmetic semantics:** `zend_add_will_overflow()`, `zend_sub_will_overflow()` and others precisely detect integer overflow |
||||
|
||||
**Operator range inference example:** |
||||
|
||||
```c |
||||
// ADD: result range |
||||
min = OP1_MIN() + OP2_MIN() |
||||
max = OP1_MAX() + OP2_MAX() |
||||
overflow = OP1_RANGE_OVERFLOW() || OP2_RANGE_OVERFLOW() |
||||
|| zend_add_will_overflow(OP1_MAX(), OP2_MAX()) |
||||
|
||||
// Result type: if overflow is true, add MAY_BE_DOUBLE to the type |
||||
// (PHP int overflow automatically converts to float) |
||||
``` |
||||
|
||||
### `update_type_info` per Opcode |
||||
|
||||
`_zend_update_type_info()` is a huge switch that precisely computes the result type and range for each Zend opcode. For example, `ZEND_ASSIGN_DIM` (array assignment) updates not only the type of the assigned element, but also the type of the array as a whole, considering MAY_BE_PACKED_GUARD (packed array guard) and reference count inference. |
||||
|
||||
### AOT Takeaways |
||||
|
||||
1. **Bitmask type system:** It is the most suitable lightweight type representation for the AOT compiler. AOT currently uses string types (`TYPE_INT = 'int'`), which cannot efficiently represent compound types like "may be int or string". Bitmasks provide O(1) union/intersect/test operations. |
||||
|
||||
2. **Range inference:** Can select the optimal integer type for C++ code generation (`int32_t` vs `int64_t` vs `BigInt`), avoiding unnecessary BigInt allocations. |
||||
|
||||
3. **Overflow tracking:** Precisely determine when conversion from int64 to float/BigInt is needed, inserting conversion code only when overflow is actually possible. |
||||
|
||||
4. **Per-opcode type update table:** The design of `_zend_update_type_info()` maps directly onto AOT's Rule system — each opcode corresponds to a Rule responsible for outputting the result type of that operation. |
||||
|
||||
--- |
||||
|
||||
## 4. SCCP (Sparse Conditional Constant Propagation) |
||||
|
||||
### Core Design |
||||
|
||||
SCCP performs both **constant propagation** and **conditional constant folding** simultaneously, and can also eliminate unreachable code (no separate dead code elimination pass needed). |
||||
|
||||
Implemented in `sccp.c`, based on `scdf.h` (the SCDF framework). |
||||
|
||||
### Value Lattice |
||||
|
||||
``` |
||||
TOP (undefined) |
||||
/ | \ |
||||
C1 C2 C3 (constant values) |
||||
\ | / |
||||
BOT (overdefined = not constant) |
||||
``` |
||||
|
||||
- TOP: the value of this variable is not yet known (optimistic assumption) |
||||
- BOT: this variable is known not to be a constant |
||||
- Constant value: the exact value is known |
||||
|
||||
### Key Algorithm Points (from the comments at sccp.c:30-74) |
||||
|
||||
**The `meet` operation (merging of φ functions):** |
||||
- BOT + any = BOT |
||||
- TOP + any = any |
||||
- C_i + C_i = C_i (two identical constants) |
||||
- C_i + C_j = BOT (two different constants) |
||||
|
||||
**Instruction evaluation:** |
||||
- Any operand is BOT → result is BOT (exception: op1 of ASSIGN) |
||||
- Instructions that can never be evaluated → BOT |
||||
- Any operand is TOP → result is TOP |
||||
- All operands are known constants → attempt compile-time evaluation → return constant value on success, BOT on failure |
||||
|
||||
**Branch feasibility determination:** |
||||
- Branch on BOT → all successors are feasible |
||||
- Branch on TOP → no successor is infeasible (wait for more information) |
||||
- Branch on a known constant → only the branch that satisfies the condition is feasible |
||||
|
||||
### The SCDF Framework (`scdf.h`) |
||||
|
||||
SCCP is built on top of the SCDF (Sparse Conditional Data Flow) framework, a generic sparse conditional data-flow analysis engine: |
||||
|
||||
```c |
||||
typedef struct _scdf_ctx { |
||||
zend_op_array *op_array; |
||||
zend_ssa *ssa; |
||||
zend_bitset instr_worklist; // Instructions to process |
||||
zend_bitset phi_var_worklist; // Phi/SSA variables to process |
||||
zend_bitset block_worklist; // Blocks to process |
||||
zend_bitset executable_blocks; // Executable blocks |
||||
zend_bitset feasible_edges; // Feasible edges |
||||
|
||||
struct { |
||||
void (*visit_instr)(...); // Process an instruction |
||||
void (*visit_phi)(...); // Process a φ function |
||||
void (*mark_feasible_successors)(...); // Mark feasible successors |
||||
} handlers; |
||||
} scdf_ctx; |
||||
``` |
||||
|
||||
**Usage pattern:** SCCP implements `visit_instr` (constant evaluation), `visit_phi` (the meet operation), and `mark_feasible_successors` (branch feasibility). Type inference also uses a similar worklist propagation algorithm. |
||||
|
||||
**Generic worklist mechanism:** |
||||
```c |
||||
// When a variable's value changes, add all its uses to the worklist |
||||
static inline void scdf_add_to_worklist(scdf_ctx *scdf, int var_num) { |
||||
const zend_ssa_var *var = &ssa->vars[var_num]; |
||||
int use; |
||||
FOREACH_USE(var, use) { |
||||
zend_bitset_incl(scdf->instr_worklist, use); // Mark instructions using this variable |
||||
} |
||||
FOREACH_PHI_USE(var, phi) { |
||||
zend_bitset_incl(scdf->phi_var_worklist, phi->ssa_var); |
||||
} |
||||
} |
||||
``` |
||||
|
||||
### AOT Takeaways |
||||
|
||||
1. **The SCDF framework is the most directly reusable:** about 300 lines of C code providing a generic worklist-driven conditional data-flow engine. AOT can port it as a PHP class, reusing it across multiple optimization passes such as SCCP, type inference, and escape analysis. |
||||
|
||||
2. **The TOP/BOT lattice model:** The AOT compiler can use the same lattice structure when analyzing types: |
||||
- TOP = unknown type (early in analysis) |
||||
- BOT = contradictory type (an inconsistency was found) |
||||
- Concrete value (a constant or an exact type) |
||||
|
||||
3. **Conditional branch feasibility:** SCCP's branch feasibility determination can directly help AOT eliminate unreachable branches at compile time, generating simpler C++ code. |
||||
|
||||
--- |
||||
|
||||
## 5. DCE (Dead Code Elimination) |
||||
|
||||
### Algorithm (`dce.c`) |
||||
|
||||
PHP's DCE uses an optimistic strategy: |
||||
|
||||
``` |
||||
1. Assume all instructions and φ functions are dead |
||||
2. Mark all instructions with obvious side effects as live (side-effect instructions) |
||||
3. Starting from live instructions, mark the defining instructions of their operands as live (reverse propagation along use-def chains) |
||||
4. Repeat until the worklist is empty |
||||
5. Delete all instructions still marked as dead |
||||
``` |
||||
|
||||
**The key `may_have_side_effects()` check (`dce.c:74-100`):** |
||||
|
||||
Zend opcodes are divided into three kinds: |
||||
- Never have side effects (such as ADD, CONCAT, BOOL_NOT): can be eliminated by DCE |
||||
- May produce a notice but have no essential side effect (such as DIV_BY_ZERO triggering a warning): configurable whether to eliminate |
||||
- Always have side effects (such as ECHO, THROW, ASSIGN_OBJ): must be preserved |
||||
|
||||
**Special capability:** It can eliminate "redundant modifications to non-escaping arrays/objects" and "useless array/object allocations". If an array is only built, modified, and used locally, the intermediate ASSIGN_DIM steps may be eliminated. |
||||
|
||||
### AOT Takeaways |
||||
|
||||
The AOT compiler's DCE can be more aggressive (because types are known at compile time): |
||||
|
||||
1. **Side-effect classification matrix:** Build a side-effect table for AOT's expression/statement types, precisely marking which operations must be preserved |
||||
2. **Escape-aware DCE:** Combined with escape analysis, eliminate operations on non-escaping objects — this is one of AOT's biggest optimization opportunities |
||||
3. **Control-dependence based DCE:** PHP explicitly states that its current DCE does not consider control dependence (comments at `dce.c:35-39`); AOT can perform more precise control-dependence DCE |
||||
|
||||
--- |
||||
|
||||
## 6. Escape Analysis |
||||
|
||||
### Algorithm (`escape_analysis.c`) |
||||
|
||||
Based on the classic escape analysis algorithm of Kotzmann & Mossenbock (PPPJ'05). |
||||
|
||||
**Core steps:** |
||||
|
||||
1. **Build equivalence escape sets (`zend_build_equi_escape_sets`):** Uses the Union-Find algorithm. If two SSA variables are related through a φ function or ASSIGN (same value), they belong to the same equivalence class. |
||||
|
||||
2. **Escape state propagation:** Each equivalence class has four states: |
||||
``` |
||||
ESCAPE_STATE_UNKNOWN → initial state (zero-initialized C memory) |
||||
ESCAPE_STATE_NO_ESCAPE → definitely does not escape (final goal) |
||||
ESCAPE_STATE_FUNCTION_ESCAPE → escapes to the called function (passed as argument) |
||||
ESCAPE_STATE_GLOBAL_ESCAPE → global escape (returned, assigned to a global variable, throws an exception, etc.) |
||||
``` |
||||
|
||||
3. **Monotonic state convergence:** States can only go from UNKNOWN → NO_ESCAPE/FUNCTION_ESCAPE/GLOBAL_ESCAPE, never reverse. |
||||
|
||||
4. **Apply escape information:** |
||||
- Non-escaping arrays can be allocated on the stack (no heap allocation needed) |
||||
- Non-escaping objects can avoid reference counting operations |
||||
- Non-escaping variables do not need separation (ZEND_SEPARATE) |
||||
|
||||
**Predecessor/successor edges:** Supports symbolic type aliases (SYMTABLE_ALIAS) and HTTP response header aliases (HTTP_RESPONSE_HEADER_ALIAS). |
||||
|
||||
### AOT Takeaways |
||||
|
||||
Escape analysis may be the most valuable for the AOT compiler: |
||||
|
||||
1. **Box allocation elimination:** AOT uses `Box<T>` to represent object references. Escape analysis can confirm which Boxes do not need heap allocation and can be created directly on the stack. |
||||
|
||||
2. **Reference count elimination:** Non-escaping objects can skip `php::Object::Ref()` / `php::Object::Unref()` operations. |
||||
|
||||
3. **Array stack allocation:** After escape analysis, local arrays can use a stack-based `zend_array`. |
||||
|
||||
4. **The 4-state model is very simple and effective**, and AOT can map it directly: |
||||
- ESCAPE_STATE_NO_ESCAPE → stack allocation |
||||
- ESCAPE_STATE_FUNCTION_ESCAPE → decided by the caller |
||||
- ESCAPE_STATE_GLOBAL_ESCAPE → heap allocation |
||||
|
||||
--- |
||||
|
||||
## 7. Call Graph |
||||
|
||||
### Design (`zend_call_graph.h`) |
||||
|
||||
PHP's call graph tracks bidirectional relationships, both caller → callee and callee → caller: |
||||
|
||||
```c |
||||
struct _zend_call_info { |
||||
zend_op_array *caller_op_array; // Caller |
||||
zend_op *caller_init_opline; // INIT_FCALL instruction |
||||
zend_op *caller_call_opline; // DO_FCALL instruction |
||||
zend_function *callee_func; // Called function |
||||
zend_call_info *next_caller; // Linked list: the callee's next caller |
||||
zend_call_info *next_callee; // Linked list: the caller's next callee |
||||
bool recursive; // Recursive call |
||||
bool send_unpack; // Uses SEND_UNPACK |
||||
bool named_args; // Named arguments |
||||
bool is_prototype; // May be a method overridden by a subclass |
||||
bool is_frameless; // frameless function |
||||
int num_args; |
||||
zend_send_arg_info arg_info[1]; |
||||
}; |
||||
|
||||
struct _zend_func_info { |
||||
zend_ssa ssa; // The function's own SSA |
||||
zend_call_info *caller_info; // Who called this function |
||||
zend_call_info *callee_info; // Whom this function called |
||||
zend_call_info **call_map; // Quick index from opnum to call_info |
||||
zend_ssa_var_info return_info; // Inferred return type |
||||
}; |
||||
``` |
||||
|
||||
**Key features:** |
||||
|
||||
1. **Bidirectional graph:** `caller_info` and `callee_info` are separate linked lists, supporting traversal upward (from callee to find callers) and downward (from caller to find callees) |
||||
2. **call_map:** An array indexing opnum → call_info, providing O(1) lookup of the call information for a given opcode position |
||||
3. **Return type propagation:** The callee's return_info can be propagated upward to the caller's return_info |
||||
4. **Argument type propagation:** The caller's actual argument types can be propagated downward to the callee's parameter types (for more precise function-body optimization) |
||||
|
||||
### Advanced Cross-Function Optimization (`zend_optimize_script:1626-1728`) |
||||
|
||||
``` |
||||
1. build_call_graph → build the bidirectional call graph |
||||
2. zend_optimize (per-func) → perform independent local optimization per function |
||||
3. analyze_call_graph → infer function information (recursion flags, indirect variable access, func_get_args, etc.) |
||||
4. build_call_map → build the opnum→call index for each function |
||||
5. dfa_analyze_op_array → build SSA + type inference (per-func) |
||||
6. dfa_optimize_op_array → perform SCCP + DCE + block pass based on SSA |
||||
``` |
||||
|
||||
### AOT Takeaways |
||||
|
||||
The AOT compiler's first two steps (prepare + convert) naturally build a complete symbol dependency graph. On top of this, it can add: |
||||
|
||||
1. **call_map index:** Quickly look up the callee's metadata from each call site (parameter types, return type, whether it is an inlining candidate) |
||||
2. **Bidirectional return type propagation:** AOT's return type inference is currently top-down; the call graph allows feeding the callee's known return type back to the caller |
||||
3. **Recursion flags:** `ZEND_FUNC_RECURSIVE_DIRECTLY` / `ZEND_FUNC_RECURSIVE_INDIRECTLY` are critical for inlining strategy decisions |
||||
|
||||
--- |
||||
|
||||
## 8. JIT IR Framework |
||||
|
||||
### Design |
||||
|
||||
The IR (Intermediate Representation) used by PHP JIT is a generic SSA-derived low-level intermediate representation, located in `ext/opcache/jit/ir/`. |
||||
|
||||
**Three stages of IR:** |
||||
|
||||
| Stage | Files | Purpose | |
||||
|------|------|------| |
||||
| IR builder | `ir_builder.h`, `zend_jit_ir.c` | Build IR instructions from Zend bytecode | |
||||
| IR optimizer | `ir_cfg.c`, `ir_fold.h`, `ir_gcm.c` | CFG optimization, constant folding, global code motion (GCM) | |
||||
| IR emitter | `ir_emit.c`, `ir_emit_x86.h` | Emit x86/ARM64 machine code from IR | |
||||
|
||||
**Example IR instructions: IR_ADD, IR_MUL, IR_LOAD, IR_STORE, IR_CALL, IR_GUARD, etc.** |
||||
|
||||
**JIT optimization levels (`zend_jit.h:32-37`):** |
||||
```c |
||||
#define ZEND_JIT_LEVEL_NONE 0 // JIT not enabled |
||||
#define ZEND_JIT_LEVEL_MINIMAL 1 // Minimal JIT (subroutine threading) |
||||
#define ZEND_JIT_LEVEL_INLINE 2 // Selective inline threading |
||||
#define ZEND_JIT_LEVEL_OPT_FUNC 3 // Optimize a single function based on type inference |
||||
#define ZEND_JIT_LEVEL_OPT_FUNCS 4 // Optimize based on the call tree |
||||
#define ZEND_JIT_LEVEL_OPT_SCRIPT 5 // Interprocedural analysis |
||||
``` |
||||
|
||||
**JIT trigger modes (`zend_jit.h:39-44`):** |
||||
```c |
||||
#define ZEND_JIT_ON_SCRIPT_LOAD 0 // Compile immediately when all functions are loaded |
||||
#define ZEND_JIT_ON_FIRST_EXEC 1 // Compile on first execution |
||||
#define ZEND_JIT_ON_PROF_REQUEST 2 // Compile the hottest functions based on profile data |
||||
#define ZEND_JIT_ON_HOT_COUNTERS 3 // Compile after N calls/loop iterations |
||||
#define ZEND_JIT_ON_HOT_TRACE 5 // Use tracing JIT after N calls |
||||
``` |
||||
|
||||
### AOT Takeaways |
||||
|
||||
1. **IR as an intermediate carrier for AST→C++:** AOT currently generates C++ code directly from the AST. Introducing an IR layer can: |
||||
- Perform optimization at the IR level (fold, GCM, register allocation simulation) |
||||
- Decouple the front end (PHP AST) from the back end (C++ codegen) |
||||
|
||||
2. **The constant folding table of `ir_fold.h`:** IR contains an auto-generated folding rule table (`gen_ir_fold_hash`) defining hundreds of algebraic simplification rules. AOT can adopt a similar "rule table"-driven constant folding approach. |
||||
|
||||
3. **JIT's profiling mechanism:** `hot_loop` / `hot_func` counters — AOT can embed profile data into the generated binary for PGO (Profile-Guided Optimization). |
||||
|
||||
--- |
||||
|
||||
## 9. OPcache Persistence & File Cache |
||||
|
||||
### Design |
||||
|
||||
OPcache is not just a cache — it stores the **optimized** bytecode in the cache. |
||||
|
||||
``` |
||||
Original PHP source code |
||||
→ compiled to zend_op_array (original bytecode) |
||||
→ through all Zend Optimizer passes (SSA + type inference + SCCP + DCE + ...) |
||||
→ keep only the optimized zend_op_array (discard temporary IR such as SSA) |
||||
→ zend_persist() serializes to shared memory / file cache |
||||
``` |
||||
|
||||
**`zend_persist_calc` + `zend_persist`**: two-phase serialization — |
||||
1. `_calc` computes the required shared memory size |
||||
2. `_persist` performs the actual serialization (all pointers adjusted to absolute offsets) |
||||
|
||||
**File cache (`zend_file_cache.c`):** Writes the persisted script to a file, allowing reuse across process restarts. |
||||
|
||||
### AOT Takeaways |
||||
|
||||
The current AOT compiler compiles from PHP source code every time. It can borrow the OPcache philosophy: |
||||
|
||||
1. **Cache the optimized AST/type information:** After the `convert()` stage, serialize the typed AST and load it directly on the next compilation |
||||
2. **Incremental compilation:** Recompile only changed files and their dependencies |
||||
3. **Two-phase serialization (`_calc` + `_persist`):** Compute the size first, then allocate memory/write, avoiding realloc fragmentation |
||||
|
||||
--- |
||||
|
||||
## 10. Other Notable Designs |
||||
|
||||
### zend_bitset |
||||
|
||||
PHP uses its own bitset implementation for efficient set operations. The optimizer frequently uses bitsets to represent worklists, live sets, and def/use sets. |
||||
|
||||
### zend_worklist.h |
||||
|
||||
Generic worklist iteration macros; SCCP and type inference use the same worklist mechanism. Designed as macros for inlining performance. |
||||
|
||||
### zend_arena |
||||
|
||||
An arena memory allocator used for fast allocation and bulk release of all optimizer data structures. One arena is bound to one `zend_optimizer_ctx`, and all passes share the same arena. |
||||
|
||||
### Inter-Pass Data Management |
||||
|
||||
Temporary IR such as SSA is destroyed immediately after a pass completes (via arena free), keeping only the optimized results in the final `zend_op_array`. This guarantees memory efficiency. |
||||
|
||||
--- |
||||
|
||||
## Recommended Adoption Order (for the AOT compiler) |
||||
|
||||
``` |
||||
Phase 1: Pass Pipeline |
||||
└── Define the AotPass enum + Pipeline runner, pluggable pass architecture |
||||
|
||||
Phase 2: Bitmask type system |
||||
└── Borrow Zend's type mask design, replacing the current string type constants |
||||
└── Map directly to C++ uint32_t constants |
||||
|
||||
Phase 3: Type inference rules |
||||
└── Each AST node/opcode corresponds to an update_type_info |
||||
└── Implement in combination with the Rule system (#4 design) |
||||
|
||||
Phase 4: SCCP + DCE |
||||
└── Constant propagation + dead code elimination based on the SCDF framework |
||||
└── Can eliminate redundant expressions before C++ code generation |
||||
|
||||
Phase 5: Escape analysis |
||||
└── Union-Find equivalence escape sets + 4-state propagation |
||||
└── Used for Box allocation elimination and reference count elimination |
||||
|
||||
Phase 6: Call graph cross-function optimization |
||||
└── Add call_map + type propagation on top of the existing symbol dependency graph |
||||
``` |
||||
|
||||
Each layer can be implemented independently and immediately bring benefits to existing code generation. |
||||
@ -0,0 +1,559 @@ |
||||
# PHPStan Design Analysis: Patterns and Modules Portable to the AOT Compiler |
||||
|
||||
This document analyzes the architecture of the PHPStan project (`projects/phpstan-src/`) and identifies design patterns and modules that can be introduced into the AOT compiler. |
||||
|
||||
--- |
||||
|
||||
## Overview: Migratable Designs Across Four Priorities |
||||
|
||||
| Priority | Design / Module | Implementation Size | Prerequisites | Benefit | |
||||
|--------|----------|---------|---------|------| |
||||
| P1 | Type object hierarchy (#1) | ~1000 lines | None | Qualitative leap in type inference precision | |
||||
| P1 | TrinaryLogic (#2) | ~80 lines | None | Type queries no longer return wrong answers | |
||||
| P1 | TypeCombinator (#3) | ~300-400 lines | #1 | Eliminate scattered type string concatenation | |
||||
| P2 | Rule system (#4) | Interface + Registry ~80 lines | Mostly independent | Architectural modularity, easy to test and extend | |
||||
| P2 | Collector two-phase analysis (#5) | ~200 lines | #4 | Cross-file global optimization | |
||||
| P2 | Extension registration mechanism (#6) | ~100 lines | #4 | Framework-level plugin system | |
||||
| P3 | TypeSpecifier / type narrowing (#7) | ~200 lines | #1, #2 | if/else branch type refinement | |
||||
| P3 | Immutable Scope (#8) | ~300 lines | #1, #2 | Expression-level type tracking | |
||||
| P4 | PHPDoc Pipeline (#9) | Reuse phpstan/phpdoc-parser | #1 | Leverage existing mature parser | |
||||
| P4 | NeverType (#10) | Contained in #1 | #1 | Dead code elimination, contradictory type detection | |
||||
|
||||
--- |
||||
|
||||
## 1. Object-Oriented Type System (Replacing String-Based Type Representation) |
||||
|
||||
### Current State |
||||
|
||||
The AOT compiler represents types with string constants: |
||||
|
||||
```php |
||||
const TYPE_INT = 'int'; |
||||
const TYPE_FLOAT = 'float'; |
||||
const TYPE_STR = 'string'; |
||||
// Compound types use string concatenation |
||||
// 'int|string', 'string|null' |
||||
``` |
||||
|
||||
This forces type operations (merging, comparison, querying) to manually parse and concatenate strings. |
||||
|
||||
### PHPStan's Approach |
||||
|
||||
PHPStan has a `Type` interface defining roughly 100 methods. Each type is a class: |
||||
|
||||
``` |
||||
Type (interface) |
||||
├── StringType |
||||
├── IntegerType |
||||
├── FloatType |
||||
├── BooleanType |
||||
├── NullType |
||||
├── MixedType (top type, with subtracted type support) |
||||
├── NeverType (bottom type, no possible values) |
||||
├── VoidType |
||||
├── ArrayType |
||||
├── ObjectType |
||||
├── CallableType |
||||
├── UnionType (A|B|C, holding a flat list<Type>) |
||||
├── IntersectionType (A&B&C) |
||||
├── ConstantStringType, ConstantIntegerType, ... |
||||
├── IntegerRangeType |
||||
└── Accessory*Type (for refinement inside IntersectionType) |
||||
``` |
||||
|
||||
**Core design principle: never use `instanceof` to determine type identity.** You must use `is*()` methods: |
||||
|
||||
```php |
||||
// Wrong — misses UnionType and IntersectionType |
||||
$type instanceof StringType |
||||
|
||||
// Correct — UnionType delegates to inner types and combines the results |
||||
$type->isString()->yes() |
||||
``` |
||||
|
||||
**Key sub-interfaces:** |
||||
|
||||
- `CompoundType`: marks types that need two-way type comparison (UnionType, IntersectionType). Adds `isAcceptedBy()`, `isSubTypeOf()` methods, implementing a double-dispatch protocol. |
||||
- `SubtractableType`: supports set difference operations (e.g. `mixed~null`). |
||||
|
||||
### The isSuperTypeOf / accepts Double-Dispatch Protocol |
||||
|
||||
This is the most central design pattern of PHPStan's type system: |
||||
|
||||
``` |
||||
Simple type (StringType)::isSuperTypeOf(Type $otherType): |
||||
if $otherType is a CompoundType: |
||||
return $otherType->isSubTypeOf($this) // reverse delegation |
||||
// own logic ... |
||||
return No |
||||
|
||||
Compound type (UnionType)::isSubTypeOf(Type $otherType): |
||||
foreach innerType in $this->types: |
||||
results[] = $otherType->isSuperTypeOf(innerType) |
||||
return extremeIdentity(results) // ALL must be subtype |
||||
``` |
||||
|
||||
**Key point:** simple types do not need to understand compound type semantics. Adding a new compound type does not require modifying any simple type's comparison logic. |
||||
|
||||
### AOT Adoption Recommendation |
||||
|
||||
The AOT compiler does not need PHPStan's full complexity. The first version only needs these concrete types: |
||||
|
||||
``` |
||||
IntegerType, FloatType, StringType, BoolType, |
||||
NullType, MixedType, NeverType, ArrayType, ObjectType, UnionType |
||||
``` |
||||
|
||||
Not needed: IntersectionType, AccessoryType, TemplateType, Constant*Type, IntegerRangeType, EnumType. |
||||
|
||||
--- |
||||
|
||||
## 2. TrinaryLogic (Three-Valued Logic) |
||||
|
||||
### Current State |
||||
|
||||
The AOT compiler uses boolean to determine type properties. But the `mixed` type means "could be string, could be int", and boolean cannot express this uncertainty. |
||||
|
||||
### PHPStan's Approach |
||||
|
||||
```php |
||||
class TrinaryLogic { |
||||
public function yes(): bool; |
||||
public function no(): bool; |
||||
public function maybe(): bool; |
||||
|
||||
public static function createYes(): self; |
||||
public static function createNo(): self; |
||||
public static function createMaybe(): self; |
||||
|
||||
public function and(self ...$others): self; |
||||
public function or(self ...$others): self; |
||||
public function extremeIdentity(self ...$others): self; // ALL yes → yes; ALL no → no |
||||
public function maxMin(self ...$others): self; // ANY yes → yes; ALL no → no |
||||
} |
||||
``` |
||||
|
||||
Usage example: |
||||
|
||||
```php |
||||
// MixedType::isString() → maybe (mixed may be string) |
||||
// IntegerType::isString() → no |
||||
// UnionType(int|string)::isString() → maybe |
||||
|
||||
$type->isString()->yes() // definitely a string |
||||
$type->isString()->no() // definitely not a string |
||||
$type->isString()->maybe() // uncertain |
||||
``` |
||||
|
||||
### AOT Adoption Recommendation |
||||
|
||||
Port directly, about 80 lines of code, no external dependencies. Replace boolean with TrinaryLogic for all type query methods. |
||||
|
||||
--- |
||||
|
||||
## 3. TypeCombinator — The Type Normalization Engine |
||||
|
||||
### Problem to Solve |
||||
|
||||
Forbid directly calling `new UnionType(...)`. All union/intersect/remove operations must go through TypeCombinator to ensure the type representation is always normalized. |
||||
|
||||
### Three Core Operations |
||||
|
||||
#### `union(Type ...$types): Type` |
||||
|
||||
**Algorithm flow:** |
||||
|
||||
1. **Fast path**: 0 arguments → `NeverType`; 1 argument → return directly; 2 arguments check `never`/`mixed`/identical objects |
||||
2. **Flattening**: `union(A, union(B, C), D)` → expands to `[A, B, C, D]` |
||||
3. **Filter NeverType**: `union(int, never, string)` → `[int, string]` |
||||
4. **Category extraction**: divide types into scalar/array/enum/integerRange/generic five categories, process in batches |
||||
5. **Scalar resolution**: `ConstantIntegerType(3) | IntegerType` → `IntegerType`; `true | false` → `BooleanType` |
||||
6. **Pairwise comparison**: |
||||
- Adjacent `IntegerRangeType` intervals merge: `int<0,5> | int<3,10>` → `int<0,10>` |
||||
- Subtype/supertype elimination: `Foo extends Bar` ⇒ `Foo | Bar` = `Bar` |
||||
- `int[] | string[]` → `(int|string)[]` |
||||
7. **Wrap-up**: 0 → NeverType; 1 → return directly; otherwise `new UnionType(array_values($types), true)` |
||||
|
||||
#### `intersect(Type ...$types): Type` |
||||
|
||||
**Algorithm flow:** |
||||
|
||||
1. If there is a `NeverType` → return never directly |
||||
2. **Distributive expansion**: `A & (B|C)` → `(A&B) | (A&C)`, then recurse on each term |
||||
3. **Flattening**: `A & (B & C)` → expands to `[A, B, C]` |
||||
4. **Pairwise two-way comparison**: |
||||
- `IntegerType & ConstantIntegerType(5)` → `ConstantIntegerType(5)` (Child & Parent = Child) |
||||
- `int & string` → `NeverType` (contradiction) |
||||
- SubtractableType difference handoff |
||||
5. **Contradiction detection**: `isSuperTypeOf` returns `no` → `NeverType` |
||||
|
||||
#### `remove(Type $fromType, Type $typeToRemove): Type` |
||||
|
||||
``` |
||||
remove(int|string, string) = int |
||||
remove(int|string|null, null) = int|string |
||||
remove(int, string) = int // what to remove is not present |
||||
remove(string, string) = never // fully removed |
||||
remove(mixed, Foo) = mixed~Foo // difference type |
||||
``` |
||||
|
||||
### AOT Adoption Recommendation |
||||
|
||||
A simplified TypeCombinator (~300-400 lines) covering union/intersect/remove for the basic types AOT needs. PHPStan's complex array shape handling, accessory type propagation, IntegerRange merging, etc. are not needed. |
||||
|
||||
### Benefits |
||||
|
||||
| Scenario | Current | After TypeCombinator | |
||||
|------|------|-------------------| |
||||
| Variable assignment of `int\|int` | String `'int\|int'` | Automatically simplified to `IntegerType` | |
||||
| Return value of `mixed\|string` | Does not know how to simplify | Automatically simplified to `MixedType` | |
||||
| Merging two branch types | Manual concatenation | `union()` automatically dedupes and removes subtypes | |
||||
| `int & string` intersection | Cannot detect | Returns `NeverType` (compile error) | |
||||
|
||||
--- |
||||
|
||||
## 4. Rule-Based Analysis Pass System |
||||
|
||||
### Core Interface |
||||
|
||||
```php |
||||
/** |
||||
* @template TNodeType of Node |
||||
*/ |
||||
interface Rule |
||||
{ |
||||
/** @return class-string<TNodeType> */ |
||||
public function getNodeType(): string; |
||||
|
||||
/** @param TNodeType $node */ |
||||
public function processNode(Node $node, Scope $scope): array; |
||||
} |
||||
``` |
||||
|
||||
Each Rule declares which AST node type it cares about and what to do when that node is found. |
||||
|
||||
### Registration Mechanism |
||||
|
||||
Declare level via PHP 8 Attribute: |
||||
|
||||
```php |
||||
#[RegisteredRule(level: 0)] |
||||
final class CallMethodsRule implements Rule |
||||
{ |
||||
public function getNodeType(): string { return MethodCall::class; } |
||||
public function processNode(Node $node, Scope $scope): array { ... } |
||||
} |
||||
``` |
||||
|
||||
### Registry Implementation |
||||
|
||||
`LazyRegistry` collects all services tagged `phpstan.rules.rule` from the DI container and indexes them by the `getNodeType()` return value. |
||||
|
||||
**Key design:** `getRules($nodeType)` matches not only the exact class name, but also all parent classes and interfaces: |
||||
|
||||
```php |
||||
public function getRules(string $nodeType): array |
||||
{ |
||||
// $nodeType = MethodCall::class |
||||
// parentNodeTypes = [MethodCall, Expr, NodeAbstract, Node, ...] |
||||
// match all Rules registered on these parent classes/interfaces |
||||
$parentNodeTypes = [$nodeType] + class_parents($nodeType) + class_implements($nodeType); |
||||
// ... |
||||
} |
||||
``` |
||||
|
||||
This means a rule registered for `Node\Expr` matches **all** expression types. |
||||
|
||||
### Runtime Dispatch |
||||
|
||||
At every node during AST traversal: |
||||
|
||||
```php |
||||
$nodeType = get_class($node); |
||||
foreach ($this->ruleRegistry->getRules($nodeType) as $rule) { |
||||
$ruleErrors = $rule->processNode($node, $scope); |
||||
// transform and collect errors ... |
||||
} |
||||
``` |
||||
|
||||
Extremely concise — no switch, no if-else chains. |
||||
|
||||
### Collector Two-Phase Analysis |
||||
|
||||
The Collector interface is almost identical to Rule, but returns collected data (instead of errors): |
||||
|
||||
```php |
||||
interface Collector |
||||
{ |
||||
public function getNodeType(): string; |
||||
/** @return TValue|null */ |
||||
public function processNode(Node $node, Scope $scope); |
||||
} |
||||
``` |
||||
|
||||
**Phase 1 (per-file):** as each file is analyzed, the collector gathers data. |
||||
|
||||
**Phase 2 (global):** after all files are analyzed, create a `CollectedDataNode` wrapping all data and run the rules registered on it: |
||||
|
||||
```php |
||||
$node = new CollectedDataNode($analyserResult->getCollectedData(), $onlyFiles); |
||||
foreach ($this->ruleRegistry->getRules(CollectedDataNode::class) as $rule) { |
||||
$ruleErrors = $rule->processNode($node, $scope); |
||||
} |
||||
``` |
||||
|
||||
`CollectedDataNode::get(string $collectorType): array<string, list<TValue>>` indexes collected data by file path. |
||||
|
||||
### Concrete Application in the AOT Compiler |
||||
|
||||
| Rule | Trigger Node | Purpose | |
||||
|------|----------|------| |
||||
| `BinaryOpCodegenRule` | `Expr\BinaryOp` | Generate C++ operation code | |
||||
| `MethodCallCodegenRule` | `Expr\MethodCall` | Method call code generation, virtual/direct call determination | |
||||
| `TypeCheckInsertRule` | `Param` / `Return_` | Insert runtime type checks at function entry/exit | |
||||
| `DeadCodeEliminateRule` | `CollectedDataNode` | Cross-file analysis, remove uncalled functions | |
||||
| `DevirtualizeRule` | `CollectedDataNode` | Single-implementation virtual methods → direct calls | |
||||
| `InlineDecisionRule` | `CollectedDataNode` | Decide inlining based on call frequency and function size | |
||||
| `ConstantFoldRule` | `Expr\BinaryOp` | Compile-time constant folding | |
||||
| `BoxOptimizationRule` | `Expr\Assign` | Box escape analysis for std containers | |
||||
|
||||
**O0/O1/O2 tiers:** |
||||
|
||||
```php |
||||
#[RegisteredRule(level: 0)] // basic code generation, always required |
||||
class BinaryOpCodegenRule implements Rule { ... } |
||||
|
||||
#[RegisteredRule(level: 1)] // O1 optimization |
||||
class ConstantFoldRule implements Rule { ... } |
||||
|
||||
#[RegisteredRule(level: 2)] // O2 aggressive optimization |
||||
class InlineDecisionRule implements Rule { ... } |
||||
``` |
||||
|
||||
### AOT Adoption Recommendation |
||||
|
||||
Progressive migration, no need to rewrite the entire compiler at once: |
||||
|
||||
1. Define the `Rule` interface and `Registry` (about 80 lines of code) |
||||
2. Extract one standalone function of `parseStmts()` as the first Rule |
||||
3. Migrate the remaining switch branches step by step |
||||
4. Introduce `Collector` + `CollectedDataNode` for cross-file optimization |
||||
|
||||
The Rule system can coexist with the existing switch dispatch — let Rules first handle the nodes they can, and fall back to the original logic for the rest. |
||||
|
||||
--- |
||||
|
||||
## 5. Extension Registration Mechanism |
||||
|
||||
### PHPStan's Approach |
||||
|
||||
PHPStan has dozens of extension interfaces, registered through the DI container's service tag mechanism: |
||||
|
||||
``` |
||||
DynamicMethodReturnTypeExtension → tag: phpstan.broker.dynamicMethodReturnTypeExtension |
||||
FunctionTypeSpecifyingExtension → tag: phpstan.typeSpecifier.functionTypeSpecifyingExtension |
||||
TypeNodeResolverExtension → tag: phpstan.phpdoc.typeNodeResolverExtension |
||||
MethodsClassReflectionExtension → tag: phpstan.broker.methodsClassReflectionExtension |
||||
PropertiesClassReflectionExtension → tag: phpstan.broker.propertiesClassReflectionExtension |
||||
... |
||||
``` |
||||
|
||||
Extensions are called before the core logic and get a chance to override the default behavior: |
||||
|
||||
```php |
||||
public function resolve(TypeNode $typeNode, NameScope $nameScope): ?Type |
||||
{ |
||||
foreach ($this->extensions as $extension) { |
||||
$type = $extension->resolve($typeNode, $nameScope); |
||||
if ($type !== null) { |
||||
return $type; // extension handled it, short-circuit core logic |
||||
} |
||||
} |
||||
// core logic ... |
||||
} |
||||
``` |
||||
|
||||
### AOT Application |
||||
|
||||
Framework-specific compilation optimizations can be provided through extension plugins instead of modifying the compiler core: |
||||
|
||||
```php |
||||
interface MethodCallOptimizationExtension |
||||
{ |
||||
/** Returns optimized C++ code, or null to indicate no handling */ |
||||
public function optimize(MethodCall $call, Scope $scope): ?string; |
||||
} |
||||
``` |
||||
|
||||
--- |
||||
|
||||
## 6. TypeSpecifier — The Type Narrowing Engine |
||||
|
||||
### Problem Solved |
||||
|
||||
When the AOT compiler encounters `if ($x instanceof Foo)`, inside the branch `$x`'s type should be narrowed to `Foo`. This enables generating more efficient C++ code (directly calling Foo's methods without going through the vtable). |
||||
|
||||
### PHPStan's Approach |
||||
|
||||
`TypeSpecifier` analyzes conditional expressions and decides how to narrow types in the truthy/falsy branches: |
||||
|
||||
| Condition | Truthy narrows to | Falsey narrows to | |
||||
|------|-------------|--------------| |
||||
| `$x instanceof Foo` | `Foo` | Remove `Foo` | |
||||
| `$x === null` | `NullType` | Remove `NullType` | |
||||
| `is_array($x)` | `ArrayType` | Remove `ArrayType` | |
||||
| `$x` (truthy) | Remove falsey types | Keep only falsey types | |
||||
| `$x > 0` | `int<1, max>` | `int<min, 0>` | |
||||
|
||||
### Type Narrowing Pipeline |
||||
|
||||
``` |
||||
Conditional expression |
||||
→ TypeSpecifier::specifyTypesInCondition(scope, expr, context) |
||||
→ SpecifiedTypes { sureTypes[], sureNotTypes[] } |
||||
→ MutatingScope::filterBySpecifiedTypes(types) |
||||
→ new scope (types already narrowed) |
||||
``` |
||||
|
||||
### AOT Application |
||||
|
||||
A simplified TypeSpecifier (~200 lines) handling: |
||||
- `instanceof` narrowing |
||||
- `=== null` / `!== null` narrowing |
||||
- `is_array()`, `is_string()`, `is_int()` and other function narrowing |
||||
- `BooleanAnd`/`BooleanOr` chained narrowing |
||||
|
||||
--- |
||||
|
||||
## 7. Immutable Scope |
||||
|
||||
### Current State |
||||
|
||||
AOT's `FunctionContext` uses mutable public arrays and tracks types only by variable name: |
||||
|
||||
```php |
||||
class FunctionContext { |
||||
public array $localVars = []; // only variable name → type |
||||
public int $scopeLevel = 0; // simple nesting counter |
||||
public array $scopeLayouts = []; // ScopeContext is an empty class |
||||
public bool $inLoop = false; |
||||
public bool $inClosure = false; |
||||
} |
||||
``` |
||||
|
||||
### PHPStan's Approach |
||||
|
||||
`MutatingScope` is an **immutable** persistent data structure. Every change returns a new instance: |
||||
|
||||
``` |
||||
scope.assignVariable('x', stringType) → new scope |
||||
scope.filterByTruthyValue(instanceofExpr) → new scope (narrowed types) |
||||
scope.filterByFalseyValue(instanceofExpr) → new scope (removed types) |
||||
scope.mergeWith(elseScope) → new scope (intersection) |
||||
``` |
||||
|
||||
Tracks types by **expression string**, not just by variable name: |
||||
|
||||
``` |
||||
'$a' → ExpressionTypeHolder($a, IntegerType, Yes) |
||||
'$a[0]' → ExpressionTypeHolder($a[0], StringType, Yes) |
||||
'$a->prop' → ExpressionTypeHolder($a->prop, FooType, Yes) |
||||
'strlen($a)' → ExpressionTypeHolder(strlen($a), IntegerType, Yes) |
||||
``` |
||||
|
||||
This means assigning to `$a` invalidates the type caches for `$a[0]` and `$a->prop`. |
||||
|
||||
### AOT Adoption Recommendation |
||||
|
||||
A simplified version (~300 lines) with core capabilities: |
||||
|
||||
- Immutable scope supporting snapshots and merging |
||||
- Type tracking by expression key (at least variables and array elements) |
||||
- `TrinaryLogic` certainty tracking (the maybe state after branch merging) |
||||
|
||||
--- |
||||
|
||||
## 8. PHPDoc Pipeline |
||||
|
||||
### PHPStan's Approach |
||||
|
||||
A three-stage pipeline: |
||||
|
||||
``` |
||||
PHPDoc comment string |
||||
→ Lexer + Parser (phpstan/phpdoc-parser) |
||||
→ PhpDocNode (raw AST) |
||||
→ TypeNodeResolver::resolve(TypeNode, NameScope) |
||||
→ PHPStan Type object |
||||
``` |
||||
|
||||
`TypeNodeResolver` dispatches via a `switch` to 30+ identifier types: |
||||
|
||||
``` |
||||
'int' / 'integer' → IntegerType |
||||
'positive-int' → IntegerRangeType(1, null) |
||||
'non-empty-string' → IntersectionType[StringType, AccessoryNonEmptyStringType] |
||||
'class-string' → ClassStringType |
||||
'array' → ArrayType(MixedType, MixedType) |
||||
'list' → IntersectionType[ArrayType(int), AccessoryArrayListType] |
||||
'mixed' → MixedType(true) |
||||
'never' → NonAcceptingNeverType |
||||
... |
||||
``` |
||||
|
||||
### AOT Application |
||||
|
||||
AOT can reuse `phpstan/phpdoc-parser` to parse `@param`, `@return`, `@var` annotations, then map the type AST nodes to AOT's own Type system. `NameScope` (tracking namespace + use imports) is also a directly reusable concept. |
||||
|
||||
--- |
||||
|
||||
## 9. NeverType (Bottom Type) |
||||
|
||||
Represents a type that "cannot have any value", i.e. the empty set: |
||||
|
||||
``` |
||||
union(int, never) = int // never is the identity element of union |
||||
intersect(string, never) = never // never is the absorbing element of intersect |
||||
``` |
||||
|
||||
### AOT Application |
||||
|
||||
- Dead code elimination: an expression narrowed to NeverType → that path is unreachable |
||||
- Error propagation: contradictory type combinations produce NeverType |
||||
- `void` function returns: essentially NeverType in the return position |
||||
|
||||
--- |
||||
|
||||
## Recommended Adoption Order |
||||
|
||||
``` |
||||
Phase 1 (foundation): Type interface + concrete types + TypeCombinator + TrinaryLogic |
||||
└── Replace string-based type representation, expected to reduce scattered type string operations |
||||
|
||||
Phase 2 (modularization): Rule interface + Registry + Attribute registration |
||||
└── Break down the huge switch of parseStmts/parseExpr |
||||
|
||||
Phase 3 (analysis): TypeSpecifier + basic Immutable Scope |
||||
└── Enable if/else branch type narrowing |
||||
|
||||
Phase 4 (optimization): Collector + CollectedDataNode |
||||
└── Enable cross-file global optimization (devirtualization, dead code elimination, inline decisions) |
||||
|
||||
Phase 5 (ecosystem): Extension registration + PHPDoc Pipeline |
||||
└── Enable framework plugins and annotation-driven type information |
||||
``` |
||||
|
||||
Each layer builds on the previous one, and each layer can deliver value independently. |
||||
|
||||
--- |
||||
|
||||
## PHPStan Features That Should NOT Be Introduced |
||||
|
||||
| Feature | Reason | |
||||
|------|------| |
||||
| Generics / `@template T` | Extremely complex, AOT does not currently need it | |
||||
| IntersectionType + Accessory types | `non-empty-string` = `string & non-empty` is an over-engineered design | |
||||
| BetterReflection (static reflection) | AOT already loads files, no need to avoid runtime side effects | |
||||
| Full MutatingScope (5000+ lines) | The immutable scope pattern is worth borrowing, but 300 lines is enough | |
||||
| ConditionalExpressionHolder | Lazy narrowing of compound conditions; do basic TypeSpecifier first | |
||||
| Enum / ConstantArrayType shape | Niche, not needed initially | |
||||
@ -0,0 +1,986 @@ |
||||
# TypePHP and Python Language-Level Interop Design |
||||
|
||||
> Status: the core design is confirmed and is being implemented in phases according to `python/implementation-plan.md`. |
||||
> |
||||
> This document is the design specification for syntax, type semantics, runtime boundaries, and compatibility goals; details that are not yet confirmed continue to be maintained at the end of the document. |
||||
|
||||
## 1. Goals |
||||
|
||||
TypePHP should provide, at the language level, the ability to call Python packages from TypePHP: |
||||
|
||||
1. TypePHP imports Python modules, accesses module members, and calls Python functions and classes. |
||||
2. TypePHP operates on Python objects, including attributes, methods, subscripts, iteration, operators, and the call protocol. |
||||
3. TypePHP functions, closures, and objects can be passed as arguments to Python calls, and Python is allowed to synchronously call back into TypePHP within that dynamic call relationship. |
||||
4. The two VMs interoperate directly within the same process, without going through JSON, RPC, or subprocesses. |
||||
5. Python object identity and type information are preserved by default, avoiding unnecessary deep copies. |
||||
6. The syntax targets ordinary TypePHP/PHP developers; routine calls do not require understanding the CPython C API, the GIL, or reference counting. |
||||
7. This feature is an optional extension-level capability; projects that do not use Python syntax do not depend on phpy. |
||||
|
||||
The most important language change is the Python special root namespace. `python\module\member()` in the global namespace, or `\python\module\member()` in other namespaces, can directly access module members; `use python\module` works entirely according to PHP's ordinary namespace alias rules, and the compiler does not apply any Python-specific treatment to the `use` statement. Both forms elevate what phpy originally required writing by hand as `PyCore::import('module')` and a returned variable into a compile-time-recognizable lazy module binding. Capabilities such as attributes, methods, subscripts, iteration, argument conversion, return wrapping, and exceptions for Python objects reuse phpy's existing implementation in principle, without re-establishing a separate runtime in TypePHP. |
||||
|
||||
Non-goals: |
||||
|
||||
- Do not compile Python source code, and do not attempt to replace CPython. |
||||
- Do not promise to statically type the dynamic Python API. |
||||
- Permanently do not support Python threads, `asyncio`, or CPython subinterpreters. |
||||
- Do not generate a Python extension, and do not register TypePHP functions, classes, or modules with Python. |
||||
- Do not provide `#[PythonExport]` or any other TypePHP symbol export mechanism. |
||||
- Do not aim to be compatible with Python syntax; the goal is to let TypePHP programs call Python packages conveniently and reliably. |
||||
- Do not automatically and recursively copy arbitrary Python containers into TypePHP arrays. |
||||
|
||||
## 2. Reference Designs |
||||
|
||||
### 2.1 Mojo |
||||
|
||||
Mojo uses an unmodified CPython runtime to guarantee Python ecosystem compatibility, and wraps dynamic Python values with a unified `PythonObject`. TypePHP only borrows its embedding and object-wrapping design, not its export mechanism. |
||||
|
||||
Borrowable parts: |
||||
|
||||
- Python values remain as wrapper objects by default. |
||||
- TypePHP base values can be automatically converted when passed into Python. |
||||
- Conversion from Python values back to TypePHP native types is explicit. |
||||
- Dynamic Python values are carried by a unified proxy type. |
||||
|
||||
References: [Mojo Python interoperability](https://docs.modular.com/stable/mojo/manual/python/), [Mojo Python types](https://docs.modular.com/mojo/manual/python/types). |
||||
|
||||
### 2.2 pybind11 |
||||
|
||||
pybind11 explicitly distinguishes object ownership, return value policies, interpreter lifetime, GIL guards, positional arguments, and keyword arguments. Its experience shows that the most dangerous part of cross-language calls is not the call syntax, but object lifetime and exception paths. |
||||
|
||||
TypePHP should not expose pybind11's ownership policies to ordinary users, but the runtime must establish an equally strict internal contract. |
||||
|
||||
References: [pybind11 embedding](https://pybind11.readthedocs.io/en/stable/advanced/embedding.html), [pybind11 functions](https://pybind11.readthedocs.io/en/stable/advanced/functions.html). |
||||
|
||||
### 2.3 PyO3 |
||||
|
||||
PyO3 uses GIL tokens and Python object pointers with lifetimes, distinguishing held objects, borrowed objects, and GIL-bound objects at the type system level. |
||||
|
||||
TypePHP does not need to expose lifetime parameters to users, but phpy's C++ layer should borrow from this: every CPython API call must be able to prove the GIL is currently held, and every `PyObject*` must be clearly an owned, borrowed, or stolen reference. |
||||
|
||||
References: [PyO3 object model](https://pyo3.rs/main/doc/pyo3/), [PyO3 Python object types](https://pyo3.rs/main/types). |
||||
|
||||
## 3. The Positioning of phpy |
||||
|
||||
phpy is a runtime foundation candidate for this feature, not a validated stable dependency. |
||||
|
||||
Reusable capabilities include: |
||||
|
||||
- Initializing CPython within the ZendVM process. |
||||
- Boundary conversion between `zval` and `PyObject*`. |
||||
- Proxy objects for Python modules, objects, strings, sequences, dictionaries, sets, iterators, and callables. |
||||
- Callable proxies for TypePHP/PHP closures passed into Python. |
||||
- Basic mapping from Python exceptions to Zend exceptions. |
||||
- Infrastructure for Python synchronous calls to functions, objects, and callable proxies actively passed in by the ZendVM. |
||||
- A prototype of the GIL RAII guard. |
||||
|
||||
However, it cannot be assumed that the existing implementation is completely correct. Subsequent implementation must simultaneously review phpy, refactor the boundaries, add tests, fix bugs, and optimize performance. |
||||
|
||||
Key audit items already identified during the design phase: |
||||
|
||||
- CPython initialization, repeated initialization, shutdown ordering, and destruction of still-alive objects. |
||||
- The owned/borrowed/stolen reference rules for every CPython API. |
||||
- The `Py_INCREF/Py_DECREF` symmetry on all success and exception paths. |
||||
- GIL acquisition, reentrant calls, and the behavior of TypePHP calling Python, which calls back into TypePHP, which calls Python again. |
||||
- The conversion process has been changed so that each top-level conversion creates an independent C++ converter object; the conversion policy, recursion stack, and depth limit are all per-object state, restored by RAII, and no longer use process-level or thread-level temporary function pointers. Cross-VM callbacks and lifetime boundaries still need to continue to be audited. |
||||
- After Python exceptions are converted to Zend exceptions, whether the CPython error indicator is always correctly cleared. |
||||
- When Zend exceptions are converted to Python exceptions, preservation of the original exception type, message, and traceback. |
||||
- Whether the operator protocol is correct. For example, PHP `/` must not be mapped to Python floor division. |
||||
- Python big integers, invalid UTF-8, bytes containing NUL, recursive containers, and cyclic references. |
||||
- When Python proxies hold Zend objects, the cross-VM reference cycles that may form between the Zend GC and the CPython GC. |
||||
- Python threads, `asyncio`, and subinterpreters must be permanently and explicitly rejected, rather than producing undefined behavior. |
||||
|
||||
TypePHP dynamically calls the `PyCore`, `PyObject`, `PyDict`, and other Facades exposed by the phpy extension through the ZendVM, without directly linking `libphpy.so` and without generating any phpy C++ symbol references. Existing public names must be preserved; TypePHP does not establish a second user-visible naming system. |
||||
|
||||
Division of responsibilities: |
||||
|
||||
- phpy is responsible for all runtime concerns: CPython initialization, the GIL, reference counting, object proxies, type conversion, exceptions, and the dual-VM lifecycle. |
||||
- phpy is responsible for providing a stable, testable Zend internal class/function/object-handler API. |
||||
- TypePHP is only responsible for recognizing language syntax, static types, and evaluation order, and generating dynamic calls based on `zend_function*` and the PHPX/Zend generic object API. |
||||
- TypePHP does not directly operate on raw `PyObject*`, and does not duplicate phpy's GIL, reference counting, or exception implementation. |
||||
- When fixing runtime bugs, fix phpy first, rather than only adding patches in TypePHP's generated code to bypass them. |
||||
|
||||
Minimal adaptation principle: |
||||
|
||||
- TypePHP's core new capabilities are Python `use` resolution, module alias symbols, and corresponding code generation. |
||||
- `python\name()`, `module\name`, `module\name()`, and operator lowering should all land on phpy's Zend Facade; Python operators call the complete CPython operator protocol through the standard library `operator` module. |
||||
- For behavior that phpy has already correctly solved, only add tests and reuse it; only modify phpy when review or tests prove a bug, that implicit conversion does not conform to TypePHP rules, or that a Zend dynamic entry point is missing. |
||||
- TypePHP does not implement CPython protocol details, and does not duplicate `PyCore`, `PyObject`, or `PyModule` logic in generated code. |
||||
|
||||
## 4. Optional Extension and Runtime Detection |
||||
|
||||
Python interop is an extension-level feature, not a mandatory dependency of the TypePHP core program. |
||||
|
||||
- TypePHP generated code only depends on ZendVM/PHPX, does not include phpy headers, and does not link `libphpy.so`. |
||||
- The compiler recognizes Python syntax and retains logical `PyObject` type information, but does not check whether the phpy SDK, dynamic library, ABI, or Python module exists. |
||||
- phpy must be loaded by the runtime environment like an ordinary PHP extension and register `PyCore`, `PyObject`, and other Zend internal classes. |
||||
- The first time a Python symbol is actually used, TypePHP resolves `PyCore` and the corresponding `zend_function*` through the class map/func map. |
||||
- When phpy is not loaded, the Zend class lookup throws a catchable PHP `Error`; if not caught, it becomes a fatal error under ordinary PHP rules. |
||||
- When phpy is loaded but the Python module does not exist, `PyCore::import()` throws a `PyError` through phpy. |
||||
- When there is only `use python\sys` without actually accessing any Python symbol, no runtime resolution occurs, so no error is reported even if phpy is not installed. |
||||
|
||||
This model allows the same TypePHP binary to run Python-free paths in environments without phpy installed, and avoids TypePHP establishing a native C++ ABI dependency on phpy. |
||||
|
||||
### 4.1 TypePHP Code Isolation |
||||
|
||||
All Python-specific implementation in TypePHP must be concentrated in an independent subdirectory, tentatively: |
||||
|
||||
```text |
||||
src/Python/ |
||||
``` |
||||
|
||||
This directory is responsible for: |
||||
|
||||
- Recognition of the `python` special root namespace. |
||||
- The import/module symbol table. |
||||
- Python Zend class/method name and logical return type mapping. |
||||
- Python syntactic sugar and static return type mapping. |
||||
- C++ lowering for Python calls, attributes, subscripts, iteration, and operators. |
||||
- Python-specific diagnostics. |
||||
|
||||
The general Parser, TypeSystem, Optimizer, and Generator may only keep minimal, stable extension entry points, and should not scatter `if ($isPython...)` special cases. When the Python feature is not enabled, Python-specific analyzers are not loaded, and existing code generation paths are not changed. |
||||
|
||||
Tests are likewise organized independently; it is recommended to use: |
||||
|
||||
```text |
||||
phpunit/src/Python/ |
||||
phpunit/code/python/ |
||||
tests/compiler/python/ |
||||
``` |
||||
|
||||
The specific directory names will be confirmed during the coding plan phase, but "implementation and test isolation" is a design constraint. |
||||
|
||||
## 5. Overall Runtime Model |
||||
|
||||
The following model is adopted: |
||||
|
||||
- One ZendVM and one CPython main interpreter coexist in one process. |
||||
- CPython is initialized and shut down entirely through phpy's existing extension lifecycle; TypePHP does not establish a second initialization path. |
||||
- All Python API boundaries automatically acquire the GIL; ordinary users do not operate the GIL. |
||||
- `PyObject` and its subclasses such as `PyDict`, `PyList`, and `PyStr` hold CPython strong references. |
||||
- Copying Python proxy objects increments the reference count; destruction decrements it within a valid interpreter/GIL context. |
||||
- Borrowed references are only allowed to exist in phpy's internal short-lifetime scopes and are not exposed to TypePHP. |
||||
- TypePHP calling Python, Python synchronously calling back a callable that TypePHP passed in as an argument, and that callable calling Python again must all support synchronous reentrancy. |
||||
- Python cannot independently import TypePHP applications, nor look up TypePHP functions or types through a global registry. |
||||
|
||||
All Python objects held by TypePHP must be released before the interpreter shuts down. One must not rely on `Py_Finalize()` to automatically repair incorrect lifetimes. |
||||
|
||||
## 6. Module Names and Import Syntax |
||||
|
||||
`python` is a reserved root namespace recognized by the compiler: |
||||
|
||||
```php |
||||
python\math\sqrt(16); |
||||
python\os\path\join('/tmp', 'file.txt'); |
||||
|
||||
use python\sys; |
||||
use Python\numpy as np; |
||||
use python\numpy\linalg as linalg; |
||||
``` |
||||
|
||||
A fully qualified name does not require writing `use` first: |
||||
|
||||
```php |
||||
$root = python\math\sqrt(16); |
||||
$pi = Python\math\pi; |
||||
``` |
||||
|
||||
All segments after the Python root and before the last `\` constitute the Python module path, and the last segment is the module member. PHP's `\` is converted to Python's `.` during import. Therefore `python\os\path\join()` in the global namespace unambiguously means the `join` callable of module `os.path`. |
||||
|
||||
Python module names still strictly obey PHP's namespace resolution rules. When inside an ordinary PHP namespace, a fully qualified module name must use a leading `\`: |
||||
|
||||
```php |
||||
namespace App; |
||||
|
||||
\python\math\sqrt(16); // Python module math |
||||
python\math\sqrt(16); // ordinary PHP name App\python\math\sqrt, not a Python module |
||||
``` |
||||
|
||||
This is part of PHP syntax; `python` is not treated as an exception that bypasses the current namespace. `use python\math;` imports from the root name like any other PHP `use` declaration, so an alias shorthand can also be used inside a namespace. |
||||
|
||||
`use` is only used to shorten a fully qualified name, not a precondition for accessing a Python module: |
||||
|
||||
PHP's `use function` and `use const` apply equally, and ordinary `as` aliases are supported: |
||||
|
||||
```php |
||||
use function python\len; |
||||
use function python\math\sqrt as py_sqrt; |
||||
use const python\math\pi as py_pi; |
||||
|
||||
$length = len([1, 2, 3]); |
||||
$root = py_sqrt(16); |
||||
$pi = py_pi; |
||||
``` |
||||
|
||||
These declarations are still handled entirely by PHP name resolution. TypePHP only enters Python lowering when the final fully qualified name of a `FuncCall` or `ConstFetch` is in the root namespace `python\...`; the `use` declaration itself does not import a Python module. |
||||
|
||||
Respectively equivalent to: |
||||
|
||||
```python |
||||
import sys |
||||
import numpy as np |
||||
import numpy.linalg as linalg |
||||
``` |
||||
|
||||
In the existing phpy PHP API, semantically corresponding to: |
||||
|
||||
```php |
||||
$sys = PyCore::import('sys'); |
||||
$np = PyCore::import('numpy'); |
||||
$linalg = PyCore::import('numpy.linalg'); |
||||
``` |
||||
|
||||
`PyCore::import()` returns a `PyModule`/`PyObject` variable, and subsequent attributes and methods are accessed through that variable. `use python\module` is just an ordinary PHP namespace alias; it does not perform the import immediately, and does not generate a ZendVM class, namespace, or user-visible variable. The compiler only checks the fully qualified name resolved by PHP when processing a function call or constant read. |
||||
|
||||
When the compiler finds `module\attr` or `module\func()` in function code, it uses the same compiler structure as the existing `funcMap`: it allocates an integer ID for the actually-used fully qualified module name, generates a unified `THREAD_LOCAL` zval array, and dynamically calls `PyCore::import()` through a lazy getter. The following names are only design sketches: |
||||
|
||||
```cpp |
||||
THREAD_LOCAL zval php_python_module_map[module_count]; |
||||
|
||||
php::Object php_get_python_module(int module_id, const php::Str &module_name) |
||||
{ |
||||
zval *module = &php_python_module_map[module_id]; |
||||
if (UNEXPECTED(Z_ISUNDEF_P(module))) { |
||||
// Resolve PyCore::import through classMap/funcMap and invoke zend_function*. |
||||
php::Variant value = php::call(/* cached zend_function* */, php::ArgList{module_name}); |
||||
ZVAL_COPY(module, value.ptr()); |
||||
} |
||||
return php::Object(module); |
||||
} |
||||
``` |
||||
|
||||
Corresponding lowering: |
||||
|
||||
```text |
||||
use Python\numpy as np |
||||
-> compile-time namespace marker: np => "numpy" |
||||
-> module id allocated only when np is actually referenced |
||||
|
||||
np\version |
||||
-> php::Object(php_get_python_module(module_id, "numpy")).attr("version") |
||||
|
||||
np\array($value) |
||||
-> php::Object(php_get_python_module(module_id, "numpy")).call("array", converted($value)) |
||||
|
||||
python\numpy\array($value) |
||||
-> the same module id and lowering as np\array($value) |
||||
|
||||
python\os\path\join($left, $right) |
||||
-> php::Object(php_get_python_module(module_id, "os.path")).call("join", ...) |
||||
``` |
||||
|
||||
The same fully qualified module name is allocated only one ID across the entire TypePHP build; when the fully qualified name and any `use` alias refer to the same module, they also share that ID. If the current `.php` file only has `use python\sys` but no `sys\attr`, `sys\func()`, or other `sys` symbol access appears, the compiler does not allocate a module ID for it, the runtime does not call `import('sys')`, and no error is reported because the Python environment lacks that module. |
||||
|
||||
An unused module does not trigger any phpy runtime resolution. `tpc` only checks the syntax and alias conflicts of `use python\sys` itself, does not check the phpy SDK/ABI, and does not add phpy link dependencies. |
||||
|
||||
### 6.1 Relationship with `funcMap` |
||||
|
||||
`pythonModuleMap` reuses the overall validated pattern of `funcMap`: |
||||
|
||||
- At compile time, a `fully qualified module name -> integer ID` map is used for deduplication. |
||||
- Data declarations are generated centrally; ordinary `.cc` files only reference the extern array and getter. |
||||
- The getter initializes on first access, and subsequent accesses hit the array directly. |
||||
- IDs are only allocated for modules where member access or calls actually occur. |
||||
- Cleanup is centralized during the application/request clean phase. |
||||
|
||||
However, the two cannot mechanically use the exact same cleanup code: |
||||
|
||||
- `funcMap` stores non-owning `zend_function*` owned by the Zend function table, so it can be cleaned up directly with `memset`. |
||||
- `pythonModuleMap` stores Zend `PyModule` object zvals returned by phpy, and cannot directly `memset` over valid objects. |
||||
- Request clean must perform `zval_ptr_dtor()` on each entry and restore it to `UNDEF`, letting phpy's own Zend object destructor handle Python references and the GIL. |
||||
- On import failure, the slot stays `UNDEF`; exception values or half-initialized objects must not be cached. |
||||
|
||||
Cleanup is done by TypePHP using the ordinary Zend zval API, without calling phpy C++ symbols: |
||||
|
||||
```cpp |
||||
for (zval &module : php_python_module_map) { |
||||
if (!Z_ISUNDEF(module)) { |
||||
zval_ptr_dtor(&module); |
||||
ZVAL_UNDEF(&module); |
||||
} |
||||
} |
||||
``` |
||||
|
||||
TypePHP only releases the Zend object; its internal Python reference counting, GIL, and error state are still handled by the phpy object handler. |
||||
|
||||
### 6.2 `sys.modules` Remains the Global Source of Truth |
||||
|
||||
Python import is itself global. When the getter first calls the underlying import, CPython returns the already-loaded module from `sys.modules` or performs the first load. `pythonModuleMap` is not a second import system; it is only equivalent to the binding stored in a Python file's namespace after it executes `import numpy as np`: |
||||
|
||||
```text |
||||
php_get_python_module(id, "numpy") |
||||
-> PyModule zval binding within the TypePHP request |
||||
-> CPython sys.modules (global module identity and load state) |
||||
``` |
||||
|
||||
It avoids re-entering the Python import API on every function call, while not taking on package lookup, loading, or reload logic. Even if the same module is referenced by multiple TypePHP files under different aliases, as long as the fully qualified module name is the same, the same ID and `PyModule` Zend object zval are used. |
||||
|
||||
This binding is consistent with ordinary Python import: if Python code later deletes or replaces `sys.modules['numpy']`, the already-completed `np` binding is not automatically changed; an explicit `PyCore::import('numpy')` is handled according to the `sys.modules` state at the time of the call. |
||||
|
||||
Rules: |
||||
|
||||
- The `python` root namespace is case-insensitive; `python`, `Python`, and `PYTHON` are all recognized as the same language symbol. |
||||
- Only the root namespace is case-insensitive. Subsequent module paths, members, methods, and keyword argument names are strictly case-sensitive. |
||||
- `python\package\module\member` in the global namespace, and `\python\package\module\member` in other namespaces, are fully qualified module accesses that do not require `use` and perform lazy import on first actual execution. |
||||
- `python\...` without a leading `\` inside a namespace is a relative PHP name and must have the current namespace prepended under PHP rules; it cannot be recognized as a Python module. |
||||
- `use python\...` can only import Python modules. |
||||
- Whether the module exists can only be determined by CPython at runtime. |
||||
- `from package import *` is not supported. |
||||
- The first version does not design a separate `from package import name` syntax; members are uniformly accessed through module aliases. |
||||
- The root namespace `\python` is reserved for language interop; for example, `App\python` is still an ordinary PHP namespace. |
||||
- Module aliases must not conflict with TypePHP classes, namespace imports, or other Python module aliases in the current file. |
||||
- Users can still call `PyCore::import()` directly and save the returned `PyModule` into an ordinary variable; fully qualified names and names resolved through ordinary PHP `use` both use the `pythonModuleMap` lazy binding. |
||||
|
||||
Example: |
||||
|
||||
```php |
||||
python\len($value); // correct |
||||
Python\len($value); // correct, different root namespace casing |
||||
python\Len($value); // error, wrong casing for the Python builtin name |
||||
Python\Len($value); // error |
||||
``` |
||||
|
||||
When the resolution result is in the root namespace `\python`, it is converted by the TypePHP compiler into a Python language symbol; when it resolves to another name such as `App\python`, an ordinary PHP function or class lookup is still performed. |
||||
|
||||
## 7. Module Members |
||||
|
||||
A Python module appears in TypePHP as a namespace, not a class. Names in the module are still dynamically resolved by the Python VM as attributes. |
||||
|
||||
### 7.1 Package Variables |
||||
|
||||
Reading a Python package variable uses the PHP namespace constant syntax `module\name`: |
||||
|
||||
```php |
||||
use python\math; |
||||
use python\os; |
||||
use python\numpy as np; |
||||
|
||||
$pi = math\pi; |
||||
$environ = os\environ; |
||||
$arrayType = np\ndarray; |
||||
$directPi = python\math\pi; |
||||
$text = math\pi->__str__(); |
||||
``` |
||||
|
||||
This uses PHP's legal namespace constant expression, but TypePHP does not register it as a Zend constant and does not perform constant folding. The compiler lowers each read to a Python module attribute lookup, and the result remains a `PyObject`, so object methods can continue to be called. |
||||
|
||||
`math::pi` or `math::$pi` is not allowed for reading package variables; both are class member syntax and would incorrectly express the module as a class. When the compiler finds such legacy syntax, it produces a targeted FatalError and suggests using `math\pi` instead. |
||||
|
||||
### 7.2 Package Functions and Class Construction |
||||
|
||||
Calling a callable in a Python package uses the PHP namespace function syntax `module\name(...)`: |
||||
|
||||
```php |
||||
$a = np\array([1, 2, 3]); |
||||
$b = np\array([4, 5, 6]); |
||||
$c = np\add($a, $b); |
||||
$root = python\math\sqrt(16); |
||||
$joined = python\os\path\join('/tmp', 'file.txt'); |
||||
``` |
||||
|
||||
The compiler reads the module's `name` attribute and calls the resulting Python object. That object can be: |
||||
|
||||
- A Python function. |
||||
- A Python class, in which case the call performs that class's construction process and returns an instance. |
||||
- Another Python object implementing `__call__`. |
||||
|
||||
TypePHP does not need to, and cannot, determine from the `np\array()` syntax alone whether it is a function or a class construction; callability is determined by Python at runtime. When a member does not exist, a Python `AttributeError` is produced; when a member is not callable, a Python `TypeError` is produced; both are uniformly mapped to `PyError`. |
||||
|
||||
The first version only supports reading module attributes. PHP namespace constant syntax itself cannot be an assignment target; when writing is needed, it should be done explicitly through the Python object API: |
||||
|
||||
```php |
||||
$os = PyCore::import('os'); |
||||
python\setattr($os, 'name', $value); |
||||
``` |
||||
|
||||
## 8. Python Builtins and phpy Syntactic Sugar |
||||
|
||||
`python\name()` means calling a Python builtin: |
||||
|
||||
```php |
||||
python\print('hello'); // equivalent to PyCore::print('hello') |
||||
$length = python\len($value)->toValue()->toInt(); |
||||
$range = python\range(0, 10); |
||||
$type = python\type($value); |
||||
``` |
||||
|
||||
It is not an ordinary TypePHP namespace function. The compiler resolves the `zend_function*` corresponding to `PyCore` through the class/func map and dynamically calls it; the runtime semantics are consistent with directly writing the corresponding `PyCore` call. |
||||
|
||||
Names are strictly case-sensitive. For known wrong names in the compiler's builtin mapping, errors can be reported at compile time; failures of other dynamic builtin lookups produce a Python `AttributeError`. |
||||
|
||||
Some names are syntactic sugar for existing phpy type constructors, rather than directly calling the same-named Python builtin: |
||||
|
||||
| TypePHP syntax | Equivalent phpy API | |
||||
|---|---| |
||||
| `python\dict($array)` | `new PyDict($array)` | |
||||
| `python\list($array)` | `new PyList($array)` | |
||||
| `python\tuple($array)` | `new PyTuple($array)` | |
||||
| `python\set($array)` | `new PySet($array)` | |
||||
| `python\str($value)` | `new PyStr($value)` | |
||||
| `python\object($value)` | `new PyObject($value)` | |
||||
| `python\print(...)` | `PyCore::print(...)` | |
||||
| `python\scalar($value)` | `PyCore::scalar($value)` | |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
$dict1 = new PyDict([1, 2, 3, 4]); |
||||
$dict2 = python\dict([1, 2, 3, 4]); |
||||
``` |
||||
|
||||
The two must have exactly the same runtime semantics. Here one cannot simply forward to CPython's `dict([1, 2, 3, 4])`, because the native Python builtin would interpret the argument as a key/value pair iterable, which differs from `PyDict`'s PHP array construction rules. |
||||
|
||||
The mapping of all syntactic sugar must form a closed, tested table; it must not be guessed based on function names alone. |
||||
|
||||
This mapping also determines compile-time static types: |
||||
|
||||
```php |
||||
$list1 = new PyList(); |
||||
$list2 = python\list(); |
||||
|
||||
$dict1 = new PyDict(); |
||||
$dict2 = python\dict(); |
||||
``` |
||||
|
||||
- Both `$list1` and `$list2` are `PyList` typed objects. |
||||
- Both `$dict1` and `$dict2` are `PyDict` typed objects. |
||||
- Both forms must use the same type checking, method resolution, and Native Call optimization. |
||||
- Syntactic sugar must not degrade to `mixed`, `var`, or only the base type `PyObject`. |
||||
- Python builtin calls also obey the object-retention rule. For example, `python\len()` returns a `PyObject` wrapping a Python int; to obtain a definite type, one must first leave the Python object rules via `toValue()` (or the function entry `python\scalar()`), then use ordinary TypePHP conversion. The Python `None` result of `python\print()` also remains a `PyObject`; when used as a standalone statement, it can simply be discarded. |
||||
- Neither `PyObject::toValue()` nor `python\scalar()` is an ordinary Python builtin call; both are conversion boundaries that explicitly require exiting the Python type rules, so they return a TypePHP `var`. |
||||
- Dynamic Python module member calls uniformly return `PyObject`. |
||||
|
||||
## 9. Python Object Types |
||||
|
||||
All Python values whose static type cannot be determined at compile time are uniformly represented as: |
||||
|
||||
```php |
||||
PyObject |
||||
``` |
||||
|
||||
`PyObject` is phpy's existing public type and TypePHP's official runtime type. `python\Object` or `python\Any` will not be introduced. |
||||
|
||||
Python built-in types continue to use phpy's existing concrete proxy classes, such as `PyDict`, `PyList`, `PyTuple`, `PySet`, `PyStr`, `PyType`, `PyFn`, and `PyIter`. In this way, ordinary PHP and TypePHP users see the same type system. |
||||
|
||||
Python's `None` is also a legal Python object. Its automatic conversion rules with TypePHP `null` need to be defined separately; Python `None` cannot be represented by a null pointer. |
||||
|
||||
## 10. Object Operations |
||||
|
||||
### 10.1 Attributes and Methods |
||||
|
||||
```php |
||||
$env = os\environ; |
||||
$items = $env->items(); |
||||
$name = $object->name; |
||||
$object->name = 'new value'; |
||||
unset($object->name); |
||||
``` |
||||
|
||||
These are respectively mapped to Python's `getattr`, call, `setattr`, and `delattr` protocols. |
||||
|
||||
`PyObject` explicitly provides two PHP Facade methods, `toValue()` and `toArray()`. `toValue()` is equivalent to `PyCore::scalar()` / `python\scalar()`, recursively converting a Python value into a PHP builtin value. Its return value then uses ordinary TypePHP conversion methods to determine the type: |
||||
|
||||
```php |
||||
$pyValue = np\int64(42); // PyObject |
||||
$value = $pyValue->toValue()->toInt(); // TypePHP int |
||||
``` |
||||
|
||||
Here `toInt()` acts on the TypePHP value already returned by `toValue()`, not on the `PyObject`. |
||||
|
||||
`toArray()` only converts Python `list`, `tuple`, `set`, `dict`, and iterators. Container elements are recursively converted to PHP values; an iterator is consumed, and subsequent conversion can only obtain its remaining elements. Python types that are not convertible return an empty array. `toArray()` is also a TypePHP keyword method, but PHPX's object conversion path calls `PyObject::toArray()`; `toString()` continues to call `PyObject::__toString()` through the keyword method, and phpy does not declare `toString()` again. |
||||
|
||||
### 10.2 Subscripts |
||||
|
||||
```php |
||||
$value = $object[$key]; |
||||
$object[$key] = $value; |
||||
unset($object[$key]); |
||||
isset($object[$key]); |
||||
``` |
||||
|
||||
These are respectively mapped to the Python mapping/sequence protocol. |
||||
|
||||
`isset()` keeps PHP's emptiness semantics: it returns `false` when the key or index does not exist, and also returns `false` when the corresponding value is Python `None`. The runtime only recognizes `KeyError` / `IndexError` as "missing"; other exceptions thrown by the Python protocol must continue to be mapped to `PyError` and must not be swallowed by `isset()`. Integer subscripts of list and tuple follow Python's negative index rules. |
||||
|
||||
### 10.3 Calling Objects |
||||
|
||||
```php |
||||
$result = $callable($arg1, $arg2); |
||||
``` |
||||
|
||||
The runtime uses `PyObject_Call`. A non-callable object produces a Python `TypeError`, mapped to a TypePHP-catchable Python exception. |
||||
|
||||
### 10.4 Iteration |
||||
|
||||
```php |
||||
foreach ($pythonIterable as $value) { |
||||
// Python __iter__ / __next__ |
||||
} |
||||
``` |
||||
|
||||
The keyed form: |
||||
|
||||
```php |
||||
foreach ($pythonIterable as $index => $value) { |
||||
} |
||||
``` |
||||
|
||||
A generic Python iterator uses the TypePHP iteration ordinal starting from `0` as `$index`, and `$value` is the object produced by `__next__()`. `PyDict` is phpy's dedicated mapping wrapper; a keyed `foreach` uses PHP mapping conventions: `$index` is the dict key, and `$value` is the corresponding dict value. Python exceptions from `__iter__()` / `__next__()` must be propagated as `PyError` and must not be treated as normal end-of-iteration. |
||||
|
||||
## 11. Arguments and Keyword Arguments |
||||
|
||||
Ordinary arguments are evaluated from left to right, then Python positional args are constructed: |
||||
|
||||
```php |
||||
$model = AutoModel\from_pretrained( |
||||
'model-name', |
||||
trust_remote_code: true, |
||||
device_map: 'auto', |
||||
); |
||||
``` |
||||
|
||||
TypePHP named arguments are mapped to Python keyword arguments. Argument names are strictly case-sensitive. |
||||
|
||||
PHP/TypePHP array unpacking rules can be used to construct positional and keyword arguments, but must satisfy: |
||||
|
||||
- Integer keys produce positional arguments. |
||||
- String keys produce keyword arguments. |
||||
- Positional arguments must not appear after keyword arguments. |
||||
- Duplicate keywords produce a Python `TypeError`. |
||||
|
||||
Whether to add explicit `python\args()` / `python\kwargs()` types is left for later discussion; the first version reuses the existing call and array unpacking syntax as much as possible. |
||||
|
||||
## 12. Explicit Conversion Principle |
||||
|
||||
TypePHP does not inherit phpy's return value implicit conversion behavior at the ZendVM Facade/opcode level. At the language level it adopts the principle of "automatic conversion when arguments enter the Python boundary, Python return values remain objects, and explicit conversion when returning to TypePHP." |
||||
|
||||
Scenarios where automatic conversion is allowed must be syntactically explicit that the code is entering Python: |
||||
|
||||
- `python\name(...)`. |
||||
- Python module calls, such as `np\array(...)`. |
||||
- Calls to `PyObject` methods or callables. |
||||
- Explicit Python container construction, such as `new PyList(...)` or `python\list(...)`. |
||||
- Parameter declarations requiring `PyObject`, `PyDict`, or other phpy types. |
||||
- Mixed arithmetic expressions composed of `PyObject` and TypePHP values. |
||||
|
||||
Within these call boundaries, all argument expressions are first strictly evaluated in TypePHP left-to-right order, then converted into objects that Python can accept. TypePHP scalars are converted to the corresponding Python scalars; TypePHP arrays are recursively converted to Python list/dict, a process that produces deep copies. This must not spread into a global implicit conversion in ordinary TypePHP expressions that do not contain Python objects. |
||||
|
||||
"Automatic conversion of all arguments" only applies to TypePHP types explicitly supported by the conversion table; resource or other values without a Python representation must throw a clear type error, not silently convert or pass invalid pointers. |
||||
|
||||
Implicit conversion is not allowed in the following scenarios: |
||||
|
||||
- Assigning a `PyObject` directly to `int`, `float`, `bool`, `string`, or `array`. |
||||
- Implicitly deep-copying a Python container into a TypePHP array. |
||||
- Arbitrarily turning a Python object into a TypePHP scalar due to arithmetic, comparison, or string context. |
||||
- Changing the static type of a TypePHP variable based on the runtime Python type. |
||||
|
||||
`echo $pyObject` can continue to be compatible with the existing `PyObject::__toString()`, but this only belongs to the output protocol and cannot be treated by the compiler as a general string implicit conversion. |
||||
|
||||
## 13. TypePHP to Python Conversion |
||||
|
||||
The Python call boundary allows the following automatic conversions: |
||||
|
||||
| TypePHP | Python | Semantics | |
||||
|---|---|---| |
||||
| `null` | `None` | singleton, not an empty `PyObject*` | |
||||
| `bool` | `bool` | value conversion | |
||||
| `int` | `int` | Python arbitrary-precision integer | |
||||
| `float` | `float` | double | |
||||
| `string` | `str` | requires valid UTF-8 | |
||||
| list array | `list` | recursive copy | |
||||
| map array | `dict` | recursive copy | |
||||
| `PyObject` and subclasses | the original object | zero copy, only passing a reference | |
||||
| TypePHP callable | Python callable proxy | Python can synchronously call back TypePHP | |
||||
| TypePHP object | Zend object proxy | object attributes are not automatically copied | |
||||
|
||||
PHP arrays use rules such as `zend_array_is_list()` to decide whether to convert to a Python `list` or `dict`. An empty array defaults to a Python `list`; if an empty dict is needed, an explicit construction API must be provided. |
||||
|
||||
Arrays and ordinary TypePHP strings may incur allocation and copying each time they enter the Python boundary. Documentation and performance diagnostics should suggest constructing and reusing native Python proxy types such as `PyDict`, `PyList`, and `PyStr` early for high-frequency calls, loop calls, or large-data scenarios, to avoid repeated deep copies. When `PyObject` and its subclasses enter the Python boundary, only a reference to the original object is passed; no content copying occurs. |
||||
|
||||
Recommended form: |
||||
|
||||
```php |
||||
// Convert only once; subsequent calls pass the same Python object. |
||||
use python\processor; |
||||
|
||||
$pyItems = python\list($items); |
||||
for ($i = 0; $i < 1000; $i++) { |
||||
processor\consume($pyItems); |
||||
} |
||||
``` |
||||
|
||||
Passing the same TypePHP container repeatedly as an argument inside a loop should be avoided, because each crossing of the Python call boundary deep-copies it again: |
||||
|
||||
```php |
||||
for ($i = 0; $i < 1000; $i++) { |
||||
processor\consume($items); |
||||
} |
||||
``` |
||||
|
||||
Strings and bytes must be distinguished. TypePHP `string` maps to Python `str` by default; binary content uses explicit `python\bytes()`. |
||||
|
||||
Recursive arrays, cyclic references, and overly deep nesting must be detected and throw exceptions, rather than recursing infinitely. |
||||
|
||||
## 14. Python to TypePHP Conversion |
||||
|
||||
### 14.1 Default Rules |
||||
|
||||
TypePHP's Python-specific call paths must turn off phpy's return value implicit conversion; all Python function, method, construction call, and operation results remain phpy objects. The static return type of dynamic calls is uniformly `PyObject`; it must not implicitly convert to a TypePHP value just because the runtime result happens to be a Python `bool`, `int`, `float`, `str`, `list`, or `dict`. |
||||
|
||||
The current implementation has the generated code dynamically call `PyCore::setOptions(['return_as_object' => true])` the first time a Python expression is actually executed. This initialization is a request-level lazy guard: only writing `use python\module` without accessing Python symbols does not trigger phpy; a constructor-only program also completes configuration before construction; request clean resets TypePHP's own guard. If phpy later provides an object-retention standalone entry point without global mode, this runtime implementation can be replaced without changing the language semantics. |
||||
|
||||
The phpy construction syntactic sugar known to the compiler still retains the precise subclasses; for example, `python\list()` returns `PyList` and `python\dict()` returns `PyDict`; these types are all `PyObject` subclasses and do not constitute return value implicit conversion. |
||||
|
||||
The phpy Zend Facade should provide mutually independent "retain Python object" and "explicitly convert to TypePHP" entry points. It must not temporarily switch by modifying process-level global function pointers or global conversion modes, otherwise nested calls, synchronous reentrancy, and exception paths may leak the wrong policy to subsequent calls. Ordinary Python calls generated by TypePHP only dynamically call the object-retention entry point; `PyObject::toValue()` and `python\scalar()` ultimately call the explicit scalar conversion entry point. |
||||
|
||||
phpy internally already uses `PythonToPhpConverter` and `PhpToPythonConverter` to implement this constraint. Each top-level conversion has an independent instance, and recursive sub-values reuse the same instance; container entry and exit are managed by RAII guards, and cyclic containers or inputs exceeding the depth limit throw a `PyError` without polluting subsequent conversions or causing a process crash. |
||||
|
||||
Reasons: |
||||
|
||||
- Preserve Python object identity and precise type. |
||||
- Avoid immediate deep copying when containers are returned. |
||||
- A Python `int` may exceed the TypePHP `int` range. |
||||
- Subclasses of Python types may override protocols and must not be forcibly expanded as base containers. |
||||
- Avoid phpy's current "partially auto-converted scalars, partially wrapped objects" behavior entering the TypePHP static type system. |
||||
|
||||
### 14.2 Explicit Conversion |
||||
|
||||
A Python object can only enter the TypePHP type rules through `toValue()`, `python\scalar()` (or the equivalent hand-written `PyCore::scalar()`): |
||||
|
||||
```php |
||||
$nativeValue1 = PyCore::scalar($value); |
||||
$nativeValue2 = python\scalar($value); // fully equivalent syntactic sugar |
||||
$nativeValue3 = $value->toValue(); |
||||
$integer = $value->toValue()->toInt(); |
||||
$float = $value->toValue()->toFloat(); |
||||
$boolean = $value->toValue()->toBool(); |
||||
$string = $value->toValue()->toString(); |
||||
$array = $value->toArray(); |
||||
``` |
||||
|
||||
Rules: |
||||
|
||||
- `toValue()` is an ordinary public method of `PyObject`, not registered as a TypePHP keyword method; internally it reuses the same converter as `PyCore::scalar()` in phpy. |
||||
- `toArray()` retains TypePHP global keyword method semantics. When PHPX performs array conversion on an object, it preferentially calls its public `toArray()`, so it enters the phpy implementation. |
||||
- After explicit conversion completes, the result fully enters TypePHP's static type, operator, and argument passing rules, no longer using Python protocols. |
||||
- Container conversion is an explicit deep conversion and detects recursive references. |
||||
- Python big integers must not silently overflow; the existing conversion rules need review before the precise mapping to TypePHP `BigInt` is determined. |
||||
- Python `str` and `bytes` must be distinguished; both must not be unconditionally converted to a TypePHP string. |
||||
- phpy is responsible for the general value conversion of `PyObject::toValue()` / `PyCore::scalar()`, and the limited container conversion of `PyObject::toArray()`. `toInt/toFloat/toBool` are post-conversion PHP values; `toString()` still calls `PyObject::__toString()` through the TypePHP keyword method. |
||||
|
||||
Existing phpy PHP users can retain compatible behavior; TypePHP calls phpy's object-retention Zend API. For this, phpy internal class methods can be refactored or added, but no C++ link dependency from TypePHP to phpy is introduced. |
||||
|
||||
## 15. Operators via the Python `operator` Module |
||||
|
||||
For `PyObject` and its subclasses: |
||||
|
||||
- `+ - * / % ** << >> & | ^` map to the corresponding functions of the Python standard library `operator` module. |
||||
- `/` maps to `operator.truediv()`, and must not map to `operator.floordiv()`. |
||||
- Python floor division temporarily uses `python\floordiv($a, $b)`, because TypePHP has no `//` operator. |
||||
- `== != < <= > >=` map to `operator.eq/ne/lt/le/gt/ge()` respectively. |
||||
- `===` / `!==` map to `operator.is_()` / `operator.is_not()` respectively. |
||||
- `if ($object)`, `!$object` use `operator.truth()`. |
||||
- Compound assignments map to `operator.iadd/isub/...()`, and the returned object updates the lvalue. |
||||
|
||||
Base mapping: |
||||
|
||||
| TypePHP | Generated dynamic call | |
||||
|---|---| |
||||
| `$a + $b` | `operator\add($a, $b)` | |
||||
| `$a - $b` | `operator\sub($a, $b)` | |
||||
| `$a * $b` | `operator\mul($a, $b)` | |
||||
| `$a / $b` | `operator\truediv($a, $b)` | |
||||
| `$a % $b` | `operator\mod($a, $b)` | |
||||
| `$a ** $b` | `operator\pow($a, $b)` | |
||||
| `$a << $b` | `operator\lshift($a, $b)` | |
||||
| `$a >> $b` | `operator\rshift($a, $b)` | |
||||
| `$a & $b` | `operator\and_($a, $b)` | |
||||
| bitwise OR | `operator\or_($a, $b)` | |
||||
| `$a ^ $b` | `operator\xor($a, $b)` | |
||||
| `-$a` | `operator\neg($a)` | |
||||
| `+$a` | `operator\pos($a)` | |
||||
| `~$a` | `operator\invert($a)` | |
||||
| `$a += $b` | `$a = operator\iadd($a, $b)` | |
||||
|
||||
All operands must be evaluated strictly from left to right. |
||||
|
||||
Even if the source code does not explicitly write `use python\operator`, when a Python operator appears, the compiler treats it as an implicit module binding used only for internal lowering, and obtains the `operator` module through the same `pythonModuleMap`. It does not inject a visible alias into the user file, so it does not conflict with the user's own `operator` class or use alias. When the user explicitly writes `use python\operator`, internal lowering and user access reuse the same module ID. |
||||
|
||||
Identity comparison calls `operator\is_()` / `operator\is_not()`. Even if the `operator\eq()` result of two objects is true, as long as they are not the same Python object, `===` is still false. |
||||
|
||||
Mixed arithmetic between Python objects and TypePHP values is allowed. As long as one side of the current operation node has static type `PyObject` or its subclass, the TypePHP expression on the other side is first fully evaluated under TypePHP rules, then the resulting value is converted to a Python object, and finally CPython executes the protocol corresponding to the current operation node. |
||||
|
||||
For example: |
||||
|
||||
```php |
||||
$result1 = $pyInt + 10; // 10 is converted to a Python int; Python performs the addition |
||||
$result2 = $pyList * getCount(); // getCount() is evaluated first, then converted to a Python int |
||||
$native = $pyInt->toValue()->toInt() + 10; // already explicitly converted to a TypePHP int; TypePHP addition is used |
||||
``` |
||||
|
||||
The result of an `operator` call is still a `PyObject`, to preserve arbitrary objects that Python custom operators may return. `===` / `!==` and conditional branches are exceptions: the Python bool results of `operator.is_/is_not/truth()` are subsequently converted through the explicit phpy conversion entry to a TypePHP `bool`. Both operands must be strictly evaluated once each from left to right, and the conversion process must not cause the expression to be executed repeatedly. |
||||
|
||||
When phpy runs as an ordinary PHP extension, it can continue to use Zend opcode handlers to provide operator overloading compatibility; TypePHP does not depend on these handlers. |
||||
|
||||
Dynamic ZendVM code has one explicitly reserved limitation: Zend compiles `-$value` / `+$value` into multiplication by `-1` / `1`, so phpy's opcode handler can no longer recognize the unary operation in the source. Therefore dynamic code retains the `$value * -1` / `$value * 1` protocol behavior and does not rewrite ordinary PHP code through a global AST hook; when a custom Python object's `__neg__()` / `__pos__()` differs from its `__mul__()`, the result may differ. TypePHP AOT still generates `operator.neg()` / `operator.pos()` according to the table above. The external user document `python.md` has already stated this limitation. |
||||
|
||||
When the TypePHP compiler recognizes static types such as `PyObject`, `PyDict`, and other phpy objects, it rewrites operators into ordinary Python module callable calls: |
||||
|
||||
```text |
||||
TypePHP operator |
||||
-> compile-time lowering |
||||
-> implicit python\operator module binding |
||||
-> operator\add/sub/... dynamic call |
||||
-> CPython complete operator protocol |
||||
``` |
||||
|
||||
This abstraction does not directly link phpy and does not go through phpy's user opcode handler, but it is not a zero-cost C++ inline operation: |
||||
|
||||
- `zend_function*` and class entries use the existing func/class map lazy cache. |
||||
- Arguments still need to be constructed as Zend values and converted by phpy into Python objects. |
||||
- Python module member lookup, GIL, CPython call, and reference counting costs still exist. |
||||
- The advantage is that the TypePHP binary only depends on ZendVM/PHPX, and phpy can be a truly optional runtime extension. |
||||
|
||||
Using the standard library `operator.add()` instead of directly calling `__add__()` reuses CPython's complete rules for `NotImplemented`, `__radd__()`, right-operand subclass priority, and so on; TypePHP does not implement reflected-operation fallback. |
||||
|
||||
The current implementation already covers binary arithmetic and bitwise operations, comparison, identity, unary operations, conditional truthiness, short-circuit logic, and compound assignment for variable, attribute, and subscript lvalues. The results of Python module functions/properties, builtins, dynamic methods, attributes, subscripts, and callables all continue to propagate the `PyObject` static type, so they can be chained or participate in subsequent Python operations. |
||||
|
||||
## 16. Exceptions |
||||
|
||||
When a Python call fails, a unified TypePHP exception type is thrown, tentatively: |
||||
|
||||
```php |
||||
PyError |
||||
``` |
||||
|
||||
The exception preserves at least: |
||||
|
||||
- The Python exception type. |
||||
- The message. |
||||
- The Python traceback object. |
||||
- The formatted traceback string. |
||||
- The original Python exception instance. |
||||
|
||||
Example: |
||||
|
||||
```php |
||||
try { |
||||
np\array('invalid')->reshape(2, 2); |
||||
} catch (PyError $error) { |
||||
echo $error->pythonType(); |
||||
echo $error->pythonTraceback(); |
||||
} |
||||
``` |
||||
|
||||
When Python synchronously calls a TypePHP callable proxy and TypePHP throws an exception, it should be converted into an ordinary Python exception, preserving the original TypePHP class name and message. That exception only propagates along the current dynamic call stack; registering a `typephp` Python module or a dedicated global exception type is not required. |
||||
|
||||
After an exception crosses a VM, the source VM's pending exception state must be cleared. Any exception conversion failure must not cause a coredump, duplicate throw, or leftover error state. |
||||
|
||||
## 17. Passing TypePHP Callables to Python |
||||
|
||||
TypePHP functions, closures, and callable objects can be automatically wrapped as Python callables: |
||||
|
||||
```php |
||||
$values = python\list([1, 2, 3]); |
||||
$result = python\map(fn (int $value): int => $value * 2, $values); |
||||
``` |
||||
|
||||
When Python calls the proxy: |
||||
|
||||
1. Python arguments are converted or wrapped into TypePHP values according to boundary rules. |
||||
2. Enter the ZendVM to call the callable. |
||||
3. The return value is converted into a Python value. |
||||
4. TypePHP exceptions are converted into Python exceptions. |
||||
|
||||
A closure proxy must hold the Zend callable to prevent it from being released while Python still references it. Cross-VM reference cycles must be explicitly detected by the runtime or provided with a predictable collection policy. |
||||
|
||||
A TypePHP callable proxy is only an argument value, not an export mechanism: only after TypePHP actively passes the proxy to Python can Python dynamically call it during the object's lifetime. TypePHP does not generate a module that Python can independently import, nor register global functions or classes. |
||||
|
||||
## 18. phpy Lifecycle and Integration |
||||
|
||||
TypePHP reuses phpy's own PHP extension entry points and lifecycle, and does not add an independent CPython bootstrap: |
||||
|
||||
1. phpy's `MINIT` initializes the shared runtime, CPython, and the `PyObject`, `PyDict`, and other Zend classes. |
||||
2. phpy's `RINIT` establishes the state needed for the current request. |
||||
3. During the request, the TypePHP program dynamically calls Python through the internal classes, methods, and object handlers that phpy registers with the ZendVM. |
||||
4. phpy's `RSHUTDOWN` releases request-level resources and proxies. |
||||
5. phpy's `MSHUTDOWN` shuts down the shared runtime and CPython after all proxies have been safely released. |
||||
|
||||
TypePHP should execute these entry points through the same mechanism as other statically or dynamically linked PHP extensions; it must not reinitialize CPython, and must not bypass phpy's lifecycle to directly call `Py_Initialize()` or `Py_Finalize()`. |
||||
|
||||
The only artifact is the main program or library with TypePHP as its entry point. No `.so` / `.pyd` importable by CPython is generated, no TypePHP modules, functions, or classes are registered with Python, and there is no `#[PythonExport]`. |
||||
|
||||
## 19. Performance Principles |
||||
|
||||
- Passing a `PyObject` only adds the necessary reference count and does not copy the Python object. |
||||
- `pythonModuleMap` only caches the `PyModule` Zend object zval that has already been bound; real loading and global identity directly reuse CPython `sys.modules`; builtin/member lookup remains simple in the first version, and a dedicated cache is only designed when benchmarks prove it necessary. |
||||
- Arguments should directly construct the array required by vectorcall, preferring the CPython vectorcall API. |
||||
- Avoid first constructing a PHP array and then having phpy convert it a second time into a Python tuple/dict. |
||||
- Converting a TypePHP array to a Python container is an explicit O(n) conversion and is not claimed to be zero-cost. |
||||
- TypePHP arrays and strings entering hot Python calls should be promoted to reusable `PyList`, `PyDict`, `PyStr`; the compiler does not cache conversion results on its own, because the original TypePHP value may have changed. |
||||
- The GIL guard should cover the minimum necessary region; correct interpreter state must be maintained during single-threaded synchronous reentrancy. |
||||
- Exception paths and normal paths must be equally tested for reference counting and memory leaks. |
||||
|
||||
## 20. Permanent Boundaries and Unsupported Capabilities |
||||
|
||||
- Python threads, including threads created by `threading` and any call entering the phpy/TypePHP bridge from a non-main thread. |
||||
- `asyncio`, Python coroutines, `async`/`await`, and cross-language event loop scheduling. |
||||
- CPython subinterpreters and the per-interpreter GIL mode. |
||||
- Python as the entry point independently loading a TypePHP program. |
||||
- Generating a Python extension or registering TypePHP functions, classes, or objects as importable Python modules. |
||||
- Runtime reflection generating TypePHP static types. |
||||
- Automatic import of `from module import *`. |
||||
- pickle/serialization of Python objects. |
||||
- Passing `PyObject` across processes. |
||||
- Python interop in the WASM target. |
||||
|
||||
Forbidden capabilities must have explicit guardrails: the compiler gives a FatalError for `threading`, `_thread`, `asyncio`, and subinterpreter APIs that can be statically recognized; phpy records the owner thread that created the runtime and refuses to enter the ZendVM bridge from other threads. Dynamic imports, reflection, or third-party packages cannot be fully recognized by the compiler, so runtime checks cannot be omitted. |
||||
|
||||
Computation threads inside third-party native packages that are completely closed and never enter the CPython API or the phpy/ZendVM bridge do not belong to the Python thread capability here; they are invisible to TypePHP and must not produce cross-thread callbacks. |
||||
|
||||
## 21. TDD and Test Gates |
||||
|
||||
The implementation and refactoring of this project must strictly follow TDD; the order must not be reversed: |
||||
|
||||
1. Write tests based on the confirmed design semantics. |
||||
2. Run the tests and confirm that they fail because the target capability is not yet implemented or an existing bug exists. |
||||
3. Write the minimal implementation that makes the tests pass. |
||||
4. Run the relevant tests and the full regression. |
||||
5. Refactor, clean up, and optimize under test protection. |
||||
6. Run the full regression, memory checks, and coverage checks again. |
||||
|
||||
It is forbidden to first complete the implementation and then add tests that only verify the current implementation details. Every bug must first have a regression test that reliably reproduces the problem. |
||||
|
||||
### 21.1 Three-Layer Mandatory Testing |
||||
|
||||
#### PHPUnit |
||||
|
||||
The TypePHP repository's PHPUnit is used to verify the compiler itself: |
||||
|
||||
- Python import and special name resolution. |
||||
- AST, symbol table, and type inference. |
||||
- C++ code generation. |
||||
- Compile-time errors and diagnostic locations. |
||||
- Compile-time diagnostics for permanently disabled capabilities, and successful code generation in environments without phpy. |
||||
- Boundary logic that does not require starting CPython. |
||||
|
||||
The phpy repository's existing PHPUnit is used to verify the ZendVM/PHP Facade and the shared Runtime: |
||||
|
||||
- Public PHP APIs such as `PyCore`, `PyObject`, and `PyDict`. |
||||
- Conversion between PHP values and Python objects. |
||||
- Mapping of Python exceptions to `PyError`. |
||||
- That the opcode handler and the Zend dynamic-call API used by TypePHP have consistent semantics. |
||||
- The object-retention call paths that TypePHP needs. |
||||
- GIL, reference counting, destruction, and exception paths. |
||||
|
||||
#### PHPT |
||||
|
||||
Used to verify end-to-end behavior of the language and runtime from the TypePHP user's perspective: |
||||
|
||||
- Import, `module\name` package variable reads, `module\name()` callable calls, and keyword arguments. |
||||
- That `use python\module as alias` and hand-written `$alias = PyCore::import('module')` are equivalent in result, exception, and object identity. |
||||
- Multiple aliases, nested modules, and repeated imports across `.cc` files. |
||||
- When there is only `use python\module` without accessing any related symbol, no helper is generated, no import is called, and the existence of that Python module is not checked. |
||||
- The same fully qualified module name is allocated only one ID across functions and across `.cc` files, and the import API is only called on first access. |
||||
- On import failure, the map slot stays `UNDEF`; after the exception is caught, the next access can retry. |
||||
- Request clean performs `zval_ptr_dtor()` on each module zval and restores it to `UNDEF`; valid Zend objects must not be directly `memset`. |
||||
- Deleting or replacing `sys.modules` entries does not change already-cached TypePHP module bindings. |
||||
- Compile-time FatalError for the legacy `module::name` / `module::$name` class member syntax, and runtime exceptions for nonexistent members and non-callable members. |
||||
- Attributes, subscripts, iteration, operators, and truthiness. |
||||
- Conversion from TypePHP arguments to Python, and explicit conversion of Python return values. |
||||
- Empty TypePHP arrays default to a Python list, and array recursive deep copying, exception abort, and repeated conversion behavior. |
||||
- Python builtins, module functions, methods, and operation results do not implicitly become TypePHP scalars. |
||||
- Explicit boundaries such as `$obj->toValue()->toInt()`, `$obj->toArray()`, `python\scalar($obj)->toInt()`, and the subsequent ordinary TypePHP conversions that restore static types and operation rules. |
||||
- Conversion from Python exceptions to TypePHP exceptions. |
||||
- When phpy is not loaded, the first Python call throws a PHP `Error`, while a merely declared unused Python `use` does not error. |
||||
- TypePHP callables being called back by Python. |
||||
- Reference counting, object destruction, and repeated calls. |
||||
- The real output of compiled programs, not just checking the generated code string. |
||||
|
||||
#### pytest |
||||
|
||||
pytest is used for regression testing of phpy's own existing Python-facing bridge; it does not mean TypePHP will generate a Python extension. It needs to verify: |
||||
|
||||
- Python calling PHP functions, objects, and callables. |
||||
- Synchronous reentrancy and the phpy module lifecycle. |
||||
- Python's holding, release, and exception mapping of Zend callable/object proxies. |
||||
- The guard that permanently forbids entering the ZendVM from Python threads. |
||||
|
||||
The three layers of testing cannot replace each other. C++/GoogleTest can cover phpy's internal reference counting, RAII, and low-level conversion, but cannot replace PHPUnit, PHPT, or pytest. |
||||
|
||||
### 21.2 Test Matrix for Each Semantic |
||||
|
||||
Each supported capability should at least consider the following dimensions: |
||||
|
||||
- The normal path. |
||||
- Error types and error messages. |
||||
- Boundary values and empty values. |
||||
- Python subclasses and dynamic protocols. |
||||
- TypePHP → Python → TypePHP reentrancy. |
||||
- Python → TypePHP → Python synchronous reentrancy initiated only by TypePHP and occurring through callable proxies. |
||||
- Normal destruction and exceptional destruction. |
||||
- Repeated execution, `sys.modules` identity, and repeated import not re-executing module code. |
||||
- Debug, Release, and supported platforms. |
||||
|
||||
Conversion tests must include: |
||||
|
||||
- `PHP_INT_MIN/PHP_INT_MAX` and out-of-range Python ints. |
||||
- `NaN`, `INF`, `-INF`, and negative zero. |
||||
- Empty strings, Unicode, invalid UTF-8, and bytes containing NUL. |
||||
- Empty list/dict, mixed keys, deep containers, recursive containers, and cyclic references. |
||||
- Identity of the same Python object after multiple wrappings. |
||||
|
||||
### 21.3 Memory and Stability Tests |
||||
|
||||
Modifications involving `PyObject*` or `zval` ownership must, in addition to functional tests, also perform: |
||||
|
||||
- PHP memory leak report. |
||||
- Python debug build/refcount checks (when the environment is available). |
||||
- ASan/UBSan builds. |
||||
- Exception injection tests covering every branch that can return early. |
||||
- Stress tests that cyclically create and destroy objects. |
||||
- Tests where cross-VM proxy objects still exist at process exit. |
||||
|
||||
Coredumps, leaks, or uncleaned pending exceptions must not be marked as "expected behavior" to bypass tests. |
||||
|
||||
### 21.4 Coverage Requirements |
||||
|
||||
- Every normative behavior in the design document must correspond to at least one test. |
||||
- New and modified bridge code needs to cover both normal and error branches. |
||||
- Overall project coverage must not decrease because of this feature. |
||||
- For GIL, reference counting, exception, and destruction code, line coverage alone is not sufficient; the branch matrix must be manually inspected. |
||||
- The final coding plan must first list the test checklist, then the implementation tasks. |
||||
|
||||
## 22. Confirmed and Pending Questions |
||||
|
||||
Confirmed: |
||||
|
||||
1. Python interop is an optional extension-level feature; TypePHP does not link or compile-time check `libphpy.so`, and if phpy is not loaded at the first actual call, Zend throws a PHP `Error`. |
||||
2. TypePHP adopts explicit conversion as much as possible and does not inherit all of phpy's implicit conversion behavior. |
||||
3. TypePHP operators are rewritten at compile time into Python standard library calls of the form `operator\add($left, $right)`, without using phpy opcode handlers and without generating phpy C++ symbol calls. |
||||
4. The `python` root namespace is case-insensitive; all Python symbols after it are case-sensitive. |
||||
5. `python` is a special language namespace handled by the compiler. |
||||
6. Runtime classes continue to use phpy's public names such as `PyObject` and `PyDict`. |
||||
7. Construction syntax such as `python\dict()` is syntactic sugar for existing phpy class constructors; `python\print()` and the like are syntactic sugar for `PyCore` APIs. |
||||
8. `new PyList()` and `python\list()` have the same `PyList` typed object type and optimization capability. |
||||
9. phpy solves runtime concerns; TypePHP only dynamically calls the phpy Facade through cached `zend_function*` and the PHPX/Zend generic object API. |
||||
10. TypePHP's Python-specific implementation and tests are placed in independent subdirectories and integrated into the general compilation flow through controlled entry points. |
||||
11. Python threads, `asyncio`, and subinterpreters are permanently forbidden and are not future compatibility targets. |
||||
12. `===` / `!==` map to Python identity `is` / `is not` respectively; `==` / `!=` use Python value comparison. |
||||
13. Only TypePHP actively calling Python is supported; no Python extension is generated, no `#[PythonExport]` is provided, and no TypePHP symbols are registered with Python. |
||||
14. The CPython and bridge lifecycle fully reuses phpy's `MINIT/RINIT/RSHUTDOWN/MSHUTDOWN` entry points. |
||||
15. Python package variables are read using the PHP namespace constant syntax `math\pi`, but at runtime a dynamic Python attribute lookup is performed; `math::pi` and `math::$pi` are incorrect class member expressions. |
||||
16. `np\array()` means reading and calling a Python package member; that member can be a function, a class, or another callable, and the specific type is determined by the Python runtime. |
||||
17. `PyObject` can be mixed with TypePHP values in operations; after TypePHP operands are converted to Python objects, the entire operation is executed by the CPython protocol, and the result remains a `PyObject`. |
||||
18. The results of Python functions, methods, class construction, and builtin calls all remain `PyObject` or known phpy subclasses; phpy return value implicit conversion is disabled. |
||||
19. `PyObject::toValue()` is an explicit scalar/container conversion method, equivalent to `python\scalar()`; `PyObject::toArray()` only accepts convertible containers and iterators, and unsupported types return an empty array. After conversion, ordinary TypePHP conversion can continue to be used, for example `$obj->toValue()->toInt()`. |
||||
20. When TypePHP calls Python, all arguments are automatically converted to Python types; TypePHP arrays are recursively deep-copied, and empty arrays default to a Python list. |
||||
21. Performance-sensitive code should reuse proxy objects such as `PyDict`, `PyList`, and `PyStr`, avoiding repeated conversion and deep copying of the same TypePHP value. |
||||
22. TypePHP's main language increment is `use python\...` and module aliases; when using aliases, phpy import is called through a lazy indexed map of the same kind as `funcMap`, and other runtime capabilities preferentially reuse phpy directly. |
||||
23. `use python\module` is handled entirely by PHP namespace resolution; when the current `.php` file does not actually access symbols that resolve to that module, no helper is generated and no runtime import is performed. |
||||
24. An ID is only allocated for a fully qualified module name when `module\attr` or `module\func()` is found; unused `use` does not occupy a map slot and does not perform import. |
||||
25. `pythonModuleMap`, like `funcMap`, is declared centrally and looked up lazily by ID; the difference is that modules are stored as owning-reference Zend object zvals and must be `zval_ptr_dtor()`-ed one by one during request clean and restored to `UNDEF`. |
||||
26. `sys.modules` is responsible for global load state and identity; `pythonModuleMap` only represents the module bindings that TypePHP has completed and cached. |
||||
27. TypePHP generated code only depends on PHPX/ZendVM; `PyCore::import()`, builtins, object methods, and conversions are all resolved as `zend_function*` dynamic calls. |
||||
28. Python operators implicitly use the `python\operator` module; the complete operator protocol is handled by CPython `operator` functions, without directly calling dunders and without TypePHP implementing reflected fallback. |
||||
|
||||
Module namespace attributes are read-only in the first version. A PHP namespace constant expression cannot be an assignment target; if write capability is added later, an explicit API should be used, and tests should be added first after the semantics are determined. |
||||
@ -0,0 +1,103 @@ |
||||
# TypePHP Python Interop Phased Implementation Plan |
||||
|
||||
> This plan takes `python/design.md` as its specification. Each phase strictly follows: first add PHPUnit/PHPT/pytest tests and confirm they fail, then implement, then run the relevant tests and the full regression. |
||||
|
||||
## Phase 1: Python module names, use, and lazy binding |
||||
|
||||
The goal is to complete a minimal runnable closed loop, without implementing operators and general conversion: |
||||
|
||||
1. Recognize `python\module\member()` / `python\module\member` in the global namespace, `\python\module\member()` / `\python\module\member` in other namespaces, the optional `use python\module` shorthand, case-insensitive root names, and case-sensitive Python subsequent names. |
||||
2. Do not build a Python-specific alias table, and do not specially handle `use`; use PHP's ordinary namespace, `use function`, `use const`, `as` alias, conflict checking, and fully qualified name resolution. Inside a namespace it must be written as `\python\module`; the relative `python\module` still resolves to a name under the current PHP namespace. |
||||
3. Allocate a module ID only when `python\module\attr`, `python\module\func()`, or their alias forms appear. |
||||
4. Generate a `pythonModuleMap` of the same kind as `funcMap`, a lazy getter, and request-clean code. |
||||
5. Dynamically call `PyCore::import()` using the Zend class/function map; do not include, link, or detect phpy. |
||||
6. Use the Zend object API to read module attributes and call module callables. |
||||
7. When phpy is not loaded, throw a PHP `Error` at the first actual use; an unused Python use does not trigger an error. |
||||
|
||||
Test order: PHPUnit code generation and diagnostic tests → PHPT runtime tests → existing compiler regression. |
||||
|
||||
Implementation status: completed. Fully qualified names and any alias share the same runtime slot per Python dotted module name; fully qualified names do not require `use`, and both syntaxes lazy-import on first actual execution. Before recognition, PHP namespace resolution is strictly applied: for example, in `namespace App`, the relative name `python\math\sqrt()` is an ordinary `App\python\math\sqrt()`, and only `\python\math\sqrt()` points to the Python root namespace. |
||||
|
||||
## Phase 2: builtins, construction syntactic sugar, and static types |
||||
|
||||
1. `python\name()` is dynamically called through the phpy Zend Facade: explicit `PyCore` methods are reused directly, and other names go through a Python `builtins` module lookup. |
||||
2. `python\list/dict/tuple/set/str/object()` maps to existing phpy Zend classes or methods. |
||||
3. `new PyList()` and `python\list()` and the like obtain the same logical static type. |
||||
4. Python call results remain `PyObject` or known phpy subclasses, turning off implicit scalar conversion on the TypePHP path. |
||||
5. Tests for missing phpy, nonexistent builtins, argument errors, and exception mapping. |
||||
|
||||
Implementation status: completed. TypePHP lazily enables phpy's `return_as_object` the first time a Python expression is actually executed; merely declaring unused Python symbols still does not trigger a runtime dependency. |
||||
|
||||
## Phase 3: argument conversion and explicit result conversion |
||||
|
||||
1. TypePHP arguments are automatically converted to Python values after left-to-right evaluation. |
||||
2. Conversion of scalars, arrays, empty arrays, nested containers, and TypePHP callables. |
||||
3. Leave the Python object rules via `$py->toValue()` or `python\scalar($py)`; when a native type is needed, continue to use ordinary TypePHP conversion, e.g. `$py->toValue()->toInt()`. Containers and iterators can directly use `$py->toArray()`. |
||||
4. Tests for deep copying, recursive containers, overflow, Unicode/bytes, and exception paths. |
||||
5. Review and refactor phpy's conversion strategy, removing global temporary conversion state that affects synchronous reentrancy. |
||||
|
||||
Implementation status: the core boundary is complete. TypePHP arguments are strictly evaluated left to right, supporting scalars, empty arrays, nested list/dict, and callables; `PyObject::toValue()` and `python\scalar()` reuse phpy's explicit conversion entry point, and `PyObject::toArray()` converts supported containers and iterators. phpy has removed the process-level conversion function pointer, replacing it with a local stateful converter, RAII recursion protection, and a 128-level depth limit, and covers invalid UTF-8, PHP self-referential arrays, and Python cyclic container error paths. The final language mapping for Python big integers and bytes remains in this phase's follow-up work. |
||||
|
||||
## Phase 4: operators |
||||
|
||||
1. Rewrite operators into dynamic calls to the Python standard library `operator` module. |
||||
2. Mixed operands are first converted to `PyObject`. |
||||
3. Strictly guarantee left-to-right evaluation, each evaluated once. |
||||
4. Use `operator.is_/is_not/truth` to implement identity and truthiness, and `iadd/isub/...` to implement compound assignment. |
||||
5. Verify that `operator` automatically handles `NotImplemented`, reflected dunders, and subclass priority. |
||||
6. Against phpy's opcode-handler behavior, fix existing issues such as `/` incorrectly mapping to floor division. |
||||
|
||||
Implementation status: completed. Binary arithmetic, bitwise operations, comparison, `===`/`!==`, unary operations, conditional truthiness, short-circuit logic, and compound assignment are all executed through the implicit `operator` module binding; `/` uses `truediv`. Mixed TypePHP operands are converted by phpy at the call boundary, results continue to remain `PyObject`, and comparison and truthiness results explicitly converge to a TypePHP `bool`. Attribute and subscript lvalues are written back through the dynamic write protocol of Phase 5. |
||||
|
||||
## Phase 5: the complete object protocol |
||||
|
||||
1. Reading, writing, and deleting Python object attributes. |
||||
2. Subscript read/write, delete, and `isset()`. |
||||
3. iterator/foreach. |
||||
4. Synchronous reentrancy of Python callables and TypePHP callable proxies. |
||||
5. keyword arguments, argument unpacking, and error semantics. |
||||
|
||||
Implementation status: completed. Dynamic attributes, unknown methods, subscripts, deletion, `isset()`, `foreach`, and callables of Python proxies all reuse phpy's Zend object protocol, without generating phpy C++ symbols. Method, attribute, subscript, and callable results continue to propagate as `PyObject`, supporting chained access and subsequent Python operations. Named arguments and unpacking reuse the unified call argument pipeline and keep left-to-right evaluation; attribute and subscript compound assignments use the returned object of `operator.i*()` to write back the original lvalue. |
||||
|
||||
phpy simultaneously completed object protocol hardening: `__set()` conversion reference release, `__unset()`, list/tuple negative indexing, list deletion, `isset()` semantics for missing keys and Python `None`, deletion and contains state checks, and iterator/count exception propagation. The related bugs are independently covered by phpy PHPUnit and TypePHP PHPT. |
||||
|
||||
## Phase 6: phpy stability and performance finishing |
||||
|
||||
1. Full audit of the CPython/ZendVM lifecycle, GIL, and owned/borrowed/stolen references. |
||||
2. Audit of Python/Zend exception state and traceback. |
||||
3. Cross-VM reference cycles, destruction, and exception injection tests. |
||||
4. ASan/UBSan, PHP leak report, Python debug build, and stress tests. |
||||
5. Benchmark dynamic Zend calls, the module map, argument conversion, and `operator` module calls; only optimize hotspots proven by data. |
||||
6. Full PHPUnit, pytest, PHPT, and existing TypePHP compiler regression. |
||||
|
||||
Implementation status: completed. The first round of CPython failure-path audits covered the construction and subscript writes of general objects, list, dict, tuple, and set, as well as sequence/set `contains()`. PHP-to-Python key/value conversion failures are now immediately mapped to `PyError`, all newly acquired references are released by scope guards; construction failure no longer leaves an unhandled CPython error indicator, and the `-1` error result of `contains()` is no longer misjudged as `true`. Paths such as invalid UTF-8, unhashable set members, and containers remaining usable after failure have phpy PHPUnit regression tests. |
||||
|
||||
The second round covered module import, exception conversion, callable checks, and the explicit iterator API. New references transferred to the caller by `PyImport_ImportModule()`, `PyErr_Fetch()`, and `PyIter_Next()` are now uniformly released after the Zend wrapper acquires an independent reference; repeated imports, Python exceptions, or explicit iterator next no longer continuously increase the reference count. Calling a non-callable Python attribute or `PyObject` reliably throws `PyError(TypeError)`, no longer silently returning `null` because `PyCallable_Check()` did not set an error indicator. `PyCore::next()` also distinguishes normal end-of-iteration from iterator exceptions. All the above paths first establish failing phpy PHPUnit regression tests, with object call behavior additionally covered by TypePHP PHPT integration. |
||||
|
||||
The third round covered conversion failures and function caching of the `PyCore` Facade. `PyCore::eval()` throws a `PyError` immediately after globals conversion fails, and `PyCore::bytes()` uses the converted `zend_string` for non-string scalars; neither dereferences a null pointer or the wrong zval union field, which would crash the process. `PyCore::next()` simultaneously releases the iterator reference produced by argument conversion. The builtin/operator function cache uses a `std::string` content key, no longer using request-level `char*` addresses as long-term keys, and also avoids the same-named dynamic call repeatedly caching and increasing Python function reference counts; calling a builtin that exists but is not callable releases the temporary reference and throws `PyError(TypeError)`. All issues are covered by independent PHPUnit tests that first fail, of which two crashes were confirmed in isolated processes with core dumps disabled (exit code 139) before being fixed. |
||||
|
||||
The fourth round covered the Python-to-PHP synchronous callback boundary. phpy converts Python keyword arguments into Zend named parameters and stops immediately after any positional or named argument conversion fails, so a PHP callable that only received part of its arguments is never executed. PHPX generates and manages Zend `arg_info` parameter name metadata for AOT native closures, so Python kwargs can be bound to TypePHP closures by name rather than depending on parameter positions or degrading to a string callable. phpy PHPUnit, PHPX unit tests, and TypePHP PHPT respectively cover conversion failure, Zend named binding, and the complete Python→TypePHP callback chain. |
||||
|
||||
The fifth round covered exceptions and ownership when Python strings cross the Zend boundary. When a Python lone surrogate character cannot be encoded as UTF-8, `phpy.String`, dynamic PHP class names, dictionary keys, `PyObject::__toString()`, and Python exception message formatting no longer use null pointers or uninitialized lengths; before the fix, the relevant isolation tests would exit with 139 or attempt to allocate abnormally large memory. `StrObject` now has an explicit valid state, and all callers must check the conversion result before accessing the pointer; stringification of exception messages is only best-effort auxiliary information, preserving the original Python error/type/value on failure and cleaning up the temporary CPython error indicator. `new_string()` also completes Zend carrier destruction registration, and fixed-length strings directly obtain a unique `zend_string` reference, eliminating the leak and uninitialized zval on the success path. |
||||
|
||||
The sixth reference audit fixed the new-reference leak in `PySequence::slice()`. After the slice is wrapped as a Zend `PyObject`, the original ownership returned by the CPython API is released while the wrapper keeps its own reference; a `sys.getrefcount()` stress test verifies that repeatedly creating and destroying slices does not continue to increase the element reference count. Slice creation failure is also converted to `PyError` before touching a null pointer. |
||||
|
||||
The seventh round covered the Python operator protocol of dynamic PHP. The phpy opcode handler now uses CPython `PyNumber_*` / `PyObject_RichCompareBool()`, `/` and `/=` use true division, compound assignment updates the Zend lvalue with the object returned by the in-place API, and correctly handles immutable Python objects, Zend reference variables, expression results, and the lvalue state after failure. `===` / `!==` use Python object identity, bool cast, `!`, and conditional branches use the Python truth protocol; new references on the PHP operand conversion, result, and exception paths are uniformly managed by RAII guards. TypePHP's `operators.phpt` runs with both ZendPHP + opcode handler and AOT + `operator` module, so it is the output-consistency gate for the two implementations. |
||||
|
||||
Unary positive/negative operations in dynamic code have been confirmed as a compatibility boundary: PHP compiles `-$value` / `+$value` into multiplication by `-1` / `1`, and the opcode handler cannot distinguish this from explicit multiplication in the source. Dynamic ZendVM code no longer attempts to rewrite the AST, but explicitly retains the `$value * -1` / `$value * 1` behavior. Python builtin numerics and common objects such as NumPy usually produce consistent results, but a custom object's `__neg__()` / `__pos__()` may differ from its `__mul__()`. AOT TypePHP retains the original AST and still lowers to `operator.neg()` / `operator.pos()` respectively, so the semantics are unaffected. The external user document `python.md` has listed this difference as a compatibility limitation. |
||||
|
||||
The performance benchmarks separately cover module properties, operator module calls, existing `PyObject` arguments, and PHP scalar argument conversion. In the current unoptimized build, module properties are about 0.7–0.9 μs/op, and operator calls about 1.8–2.5 μs/op; whether arguments are pre-wrapped as `PyObject` shows no stable difference. After attempting to eliminate the per-access reference count of the module map with an indirect zval wrapper, the A/B medians remained within the same noise range, so the more lifetime-sensitive optimization with unproven benefit was not retained. |
||||
|
||||
phpy's CMake Python-extension target also completed the out-of-tree build fix, and is compatible with the `int` ABI of older PHP and the `zend_result` ABI of newer PHP by deriving the return type from Zend's standard cast handler. Both PHP 8.1 and PHP 8.4 have completed clean-build verification. |
||||
|
||||
The memory gate uses Valgrind Memcheck. Tests disable the Zend allocator and PCRE JIT, and in a minimal independent process loop 100 times each over PHP Closure kwargs callbacks, callable PHP object kwargs callbacks, sequence slice creation/destruction, object stringification of invalid Unicode, dictionary key conversion, and exception formatting. The result is 0 invalid-access, 0 definite leak, 0 indirect leak; the 493,106 bytes retained by PHP/CPython at process exit are all still-reachable and not counted as leaks. The ASan extension cannot be safely `dlopen`-ed into the current non-ASan PHP with `RTLD_DEEPBIND` enabled, so this round uses Valgrind, which does not require recompiling PHP, as the memory checking tool. |
||||
|
||||
Phase 6 final gate results: phpy PHPUnit 135 tests / 469 assertions passed (1 existing warning, 1 environment-related skip), pytest 26/26 passed, TypePHP Python PHPT 14/14 passed, TypePHP PHPUnit 1103 tests / 2729 assertions passed; the TypePHP compiler full PHPT has 934 items in total, of which 932 PASS, 2 SKIP, 0 FAIL, 0 WARN. The dynamic operator stress test under Valgrind is 0 invalid access, 0 definite leak, 0 indirect leak. |
||||
|
||||
## Phase Gates |
||||
|
||||
- Do not start implementation before the current phase's failing tests are established. |
||||
- Do not enter the next phase before all tests of the current phase pass. |
||||
- phpy behavior changes must first add PHPUnit/pytest tests in the phpy repository. |
||||
- Every fixed bug must retain an independent regression test. |
||||
- Do not mask implementation differences by modifying third-party test expectations. |
||||
@ -0,0 +1,98 @@ |
||||
# Python Tools Submodule |
||||
|
||||
TypePHP integrates the Python IDE helper generator and the Python source converter into `tpc`. Both are located in an independent |
||||
`src/PythonTools` directory, only reusing the `tpc` command entry point, and do not enter the normal PHP preprocessing, C++ generation, and compilation pipeline. |
||||
|
||||
## Python namespace IDE helper |
||||
|
||||
```shell |
||||
./tpc --gen-python-helper math |
||||
./tpc --gen-python-helper numpy.linalg |
||||
./tpc --gen-python-helper numpy --output-dir .ide-helper |
||||
``` |
||||
|
||||
The command imports the specified Python module through PHPy, and uses the Python `inspect` API to collect functions, parameters, classes, methods, and |
||||
module attributes. The PHPy extension and the target Python module must be installed in the host environment where `tpc` is executed. |
||||
|
||||
By default, generated files are located in `ide-helper` in the current directory. `--output-dir` can replace this output root directory, supporting both |
||||
paths relative to the current directory and absolute paths: |
||||
|
||||
```text |
||||
ide-helper/python/math.php |
||||
ide-helper/python/numpy/linalg.php |
||||
ide-helper/python.php |
||||
ide-helper/PyObject.php |
||||
``` |
||||
|
||||
Each time a module helper is generated, the Python `builtins` are also scanned and the root namespace file |
||||
`python.php` is generated, providing IDE completion for builtin symbols such as `python\tuple()` and `python\len()`. This file is |
||||
regenerated according to the current Python environment. |
||||
|
||||
When a module helper is first generated, the common `PyObject.php` is also generated. It contains method hints for dynamic access, calls, |
||||
array access, iteration, and `toArray()`, `toValue()` of `PyObject`, shared by all Python module helpers. If this file |
||||
already exists, the generator keeps the original file and does not overwrite it. |
||||
|
||||
Generated content uses TypePHP's module-as-namespace form, e.g. `python\math\sqrt()`, and is compatible with the IDE name resolution of ordinary |
||||
`use`, `use function`, and `use const`. The end of the file contains `die`, which is used to explicitly |
||||
terminate the program when executed by mistake. Helpers can only be handed to the IDE for indexing; they cannot be included, nor added to a TypePHP project's sources or compilation inputs. |
||||
|
||||
`PyObject::IDE_HELPER_ONLY` is a hint constant shared by all helpers. The method bodies of non-`void` stubs use |
||||
`die(\PyObject::IDE_HELPER_ONLY)` to satisfy the IDE's control-flow checks on return types, no longer producing a "missing return |
||||
statement" diagnostic. Module attributes use namespace `const` declarations, supporting the IDE's constant completion and `use const`. |
||||
PHP 8.1 and above allow `new` in constant initializer expressions. Module attributes therefore directly use an IDE-analysis-only |
||||
`PyObject` instance as a placeholder value: |
||||
|
||||
```php |
||||
const pi = new \PyObject(); |
||||
``` |
||||
|
||||
This way the IDE precisely recognizes the constant as `PyObject`, instead of inferring a wrong type from `null`. |
||||
|
||||
The common `PyObject` helper also declares TypePHP's virtual keyword methods, including `toInt()`, `toFloat()`, |
||||
`toString()`, `toBool()`, `toStream()`, high-precision type conversions, `toObject()`, `toAny()`, and `toRef()`. |
||||
These declarations are only used for IDE completion; calls are expanded at compile time and are not entity methods of the PHPy `PyObject` runtime class. |
||||
`toArray()` and `toValue()` are still real methods provided by PHPy. |
||||
|
||||
Python class constructors explicitly call `parent::__construct()`. If a Python object defines `count()`, the helper |
||||
does not declare it again, because `PyObject::count(): int` is already used for PHP `Countable`. When the Python object's own |
||||
`count()` needs to be called, it should be written explicitly as `$object->__call('count', $arguments)`. |
||||
|
||||
PHP function/class names are case-insensitive, while Python names are case-sensitive; PHP reserved words also cannot be declared as ordinary |
||||
stub symbols. The generator reports symbols that cannot be expressed with legal PHP declarations as comments, and does not rename the Python API on its own. |
||||
The call syntax of `python\print()` is legal, but PHP forbids declaring a function named `print`, so a plain |
||||
PHP helper file cannot provide a syntax-error-free symbol declaration for it. Reserved words such as `list`, `int`, and `float` have |
||||
the same limitation. |
||||
|
||||
## Python to TypePHP |
||||
|
||||
```shell |
||||
./tpc --convert-python-to-php script.py > script.php |
||||
``` |
||||
|
||||
The converter invokes `python3` in the PATH to parse the Python AST, then outputs PHP source using the TypePHP Python namespace |
||||
syntax. Ordinary module imports are converted to namespace imports: |
||||
|
||||
```python |
||||
import math |
||||
print(math.sqrt(16)) |
||||
``` |
||||
|
||||
```php |
||||
use python\math; |
||||
|
||||
function main(): void |
||||
{ |
||||
python\print(math\sqrt(16)); |
||||
} |
||||
``` |
||||
|
||||
Currently it supports ordinary imports, functions, assignments, calls, container literals, basic operations, single comparisons, if/while/for, |
||||
lambda, and basic f-strings. Module top-level variables are converted to PHP globals to preserve the ability of functions to read module variables. |
||||
When semantics can be strictly preserved, the converter directly uses PHP native syntax: `print()` with no arguments or safely convertible arguments |
||||
generates `echo` with a newline, and `sys.exit()` with integer literal exit codes generates `exit`. `print()` with |
||||
`sep`, `end`, `file`, or `flush` arguments, and `sys.exit()` with strings or objects, |
||||
are not fully consistent with PHP behavior and remain as Python calls. |
||||
|
||||
The converter follows the principle of "reject when semantics cannot be reliably preserved". Not-yet-completed syntax such as class, async, generator, try/with, decorator, |
||||
destructuring assignment, chained comparison, nested functions, and loop-else throws an error with the |
||||
source file and line number, rather than generating PHP code that looks usable but is semantically wrong. |
||||
@ -0,0 +1,22 @@ |
||||
## Compilation |
||||
|
||||
```shell |
||||
./tpc --wasm test.php |
||||
``` |
||||
|
||||
After a successful compilation, by default only the WASI 0.2 Component `test.wasm` executable by Wasmtime is generated. WASI 0.1 is not supported. |
||||
The generated C++ source is written to `build/` by default, and can also be specified via `--build-dir <directory>`. |
||||
|
||||
## Execution |
||||
|
||||
```shell |
||||
wasmtime test.wasm |
||||
``` |
||||
|
||||
## Chrome |
||||
|
||||
```shell |
||||
./tpc --wasm=browser test.php |
||||
``` |
||||
|
||||
Browser mode additionally generates the `test.browser/` Jco module and requires `jco` to be on the `PATH`. The full browser demo is located in the repository's `examples/wasm-hello/`, and is built using the `project.yml` with `wasm: browser`. TypePHP executes in a dedicated Worker; the default filesystem resides in memory, and OPFS snapshot persistence can be explicitly enabled. Network sockets, processes, shell, and signals are explicitly not supported under the WASI target. |
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue