feat(parser): add array reference support and improve foreach handling

- Add support for array references using byRef flag in ArrayExpressionTrait
- Modify hasReference check in array parsing logic to handle mixed references
- Implement reference value parsing with proper error handling for unpacking
- Enhance foreach reference binding with improved type checking and error messages
- Add proper scope handling for foreach iterators with class context
- Update foreach value assignment to use correct reference expression syntax
- Implement structural mutation protection for std containers during iteration
- Add comprehensive test coverage for foreach edge cases and reference mutations
- Document std container modification restrictions during foreach loops
- Update yield from and foreach documentation for iterator handling differences
pull/18/head
韩天峰 1 month ago
parent 10d807bed6
commit 1403590a32
  1. 1
      docs/INCOMPATIBLE_PHP_FEATURES.md
  2. 1
      docs/PHP_INCOMPATIBILITY_CLASSIFICATION.md
  3. 12
      docs/STD_CONTAINERS.md
  4. 9
      docs/YIELD_GENERATOR.md
  5. 16
      src/Parser/ArrayExpressionTrait.php
  6. 4
      src/Parser/AssignOpTrait.php
  7. 30
      src/Parser/ForeachTrait.php
  8. 5
      src/Parser/PropertyAccessTrait.php
  9. 32
      src/Parser/StdContainerTrait.php
  10. 75
      tests/compiler/loop/foreach-array-edge-cases.phpt
  11. 112
      tests/compiler/loop/foreach-iterator-callbacks.phpt
  12. 108
      tests/compiler/loop/foreach-object-semantics.phpt
  13. 100
      tests/compiler/loop/foreach-reference-mutation.phpt
  14. 29
      tests/compiler/std-vector/foreach-element-update.phpt

@ -54,6 +54,7 @@
- `match` 的 arm condition 不能是 `match` 表达式。
- `foreach` by reference 的 value 只能是变量。
- `foreach` by reference 不支持 list destructuring。
- `std::vector`、`std::map`、`std::ordered_map` 在 `foreach` 期间禁止追加、插入、`unset()` 或整体替换;已有元素的非结构性更新仍可使用赋值运算符完成。
- 固定 native typed object property 不允许按 PHP 未初始化语义自由 `unset()`
- native 类型变量执行 `unset()` 不会产生标准 PHP 的变量删除语义。

@ -75,6 +75,7 @@ These items should be documented with the exact boundary.
| Strict function argument counts | Intentional Rule | Non-variadic functions reject extra arguments. `func_get_args()` does not implicitly make a function variadic. |
| Reserved keyword methods such as `toArray()` | Intentional Rule | Conversion keywords are resolved before ordinary object methods to keep conversion lowering static and predictable. |
| Zero-initialized fixed typed property slots | Intentional Rule / Partial | Native fixed-layout slots use their type's zero value instead of preserving every Zend uninitialized-property transition. |
| Structural mutation of `std` containers during `foreach` | Intentional Rule | Native C++ iterators may be invalidated by append, insertion, erase or whole-container replacement. TypePHP rejects these operations inside the active loop while allowing non-structural element updates. |
## Implementable but Currently Unsupported

@ -196,6 +196,18 @@ $b[] = 20;
$a = $b; // 允许,类型完全一致,执行容器 copy
```
### foreach 中修改元素
遍历 `std::vector`、`std::map` 或 `std::ordered_map` 时,可以更新已经存在的元素值,例如使用 `+=`
```php
foreach ($vector as $index => $value) {
$vector[$index] += 10;
}
```
遍历期间不能执行可能使 C++ iterator 失效的结构修改,包括追加元素、插入或覆盖 key、`unset()` 以及整体替换容器。编译器会直接报告错误。需要改变结构时,先记录待处理的 key,结束 `foreach` 后再统一修改。
## std::ordered_map
`std::ordered_map` 是有序 key-value 容器。

@ -85,14 +85,15 @@ TypePHP generator 暂不支持以下参数声明:
### Traversable 边界
`yield from` 和 TypePHP Native 对象 `foreach` 当前通过 `Iterator`/`IteratorAggregate` 接口方法驱动对象,而不是直接使用所有 Zend iterator handlers
`yield from` 与 TypePHP Native 对象 `foreach` 使用不同的底层路径。`foreach` 统一通过 PHPX `ForeachIterator` 驱动数组、普通对象和 Zend `Traversable`;`yield from` 仍由 generator 自己完成委托
这意味着:
- 用户态 `Iterator`、`IteratorAggregate` 和 Zend `Generator` 可以迭代。
- 用户态 `Iterator`、`IteratorAggregate`、Zend `Generator` 以及扩展提供的内部 `Traversable` 均通过类的 `get_iterator` handler 迭代。
- 普通对象直接遍历实时 property table,并按当前 TypePHP 类作用域执行 public、protected、private 可见性检查。
- 不读取 key 的 `foreach ($iterable as $value)` 不会调用 `Iterator::key()`
- TypePHP `yield from` 会检测 `IteratorAggregate::getIterator()` 返回自身或形成对象环,并抛出异常。
- TypePHP Native 对象 `foreach``IteratorAggregate` 展开路径尚未加入同样的环检测;`getIterator()` 返回自身或形成对象环时可能无限循环,应避免这种实现。
- 某些扩展提供的内部 `Traversable` 如果既不实现 `Iterator`,也不实现 `IteratorAggregate`,可能无法按 PHP 原生 `foreach` 的方式迭代。
- TypePHP Native `foreach``IteratorAggregate` 的展开与环检测交给 Zend iterator handler,行为与当前 PHP 运行时保持一致。
- 非法 `getIterator()` 返回值的异常类型、消息文本和栈信息可能与 ZendVM 不完全一致。
### Fiber 可观察差异

@ -29,10 +29,14 @@ trait ArrayExpressionTrait
$hasUnpack = false;
$hasVarKey = false;
$hasNextInsert = false;
$hasReference = false;
foreach ($items as $item) {
if ($item->unpack) {
$hasUnpack = true;
}
if ($item->byRef) {
$hasReference = true;
}
if ($item->key) {
if ($item->key instanceof Node\Scalar\LNumber) {
$hasIntKey = true;
@ -48,7 +52,7 @@ trait ArrayExpressionTrait
}
// 存在混合键,则需要拆分为多行插入
if ($hasUnpack or $hasVarKey or ($hasNextInsert && $hasKey) or ($hasIntKey and $hasStrKey)) {
if ($hasReference or $hasUnpack or $hasVarKey or ($hasNextInsert && $hasKey) or ($hasIntKey and $hasStrKey)) {
return $this->parseArrayMixed($node);
}
@ -211,7 +215,14 @@ trait ArrayExpressionTrait
$items = $node->items;
foreach ($items as $item) {
$this->assertExprCanBeUsedAsValue($item->value, $item->unpack ? 'array unpack value' : 'array value');
$value = $this->parseIdentifier($item->value);
if ($item->byRef) {
if ($item->unpack) {
$this->fatalError($item, 'Cannot unpack references in array literals');
}
$value = $this->convertToRef($item->value);
} else {
$value = $this->parseIdentifier($item->value);
}
if ($item->unpack) {
$this->context->beforeStmtLines[] = $this->getIndent() . $tmpVar . '.merge(' . $value . ');';
} elseif ($item->key) {
@ -226,4 +237,3 @@ trait ArrayExpressionTrait
return $tmpVar;
}
}

@ -364,6 +364,10 @@ trait AssignOpTrait
$this->fatalError($right, 'Cannot copy std container with different type');
}
if (!$this->isStdArray($leftVar)) {
$this->assertStdContainerStructureMutable($right, $leftVar);
}
return $leftVar . '_ref = ' . $this->parseStdContainerCopyExpr($right);
}

@ -95,14 +95,22 @@ trait ForeachTrait
if ($node->byRef) {
if (!$this->hasVar($valueVar)) {
$this->addLocalVar($valueVar, Type::REF);
} elseif ($this->getVarType($valueVar) !== Type::REF) {
$this->fatalError($node, 'Cannot assign value to reference of type');
} elseif ($this->getVarType($valueVar) !== Type::REF && $this->getVarType($valueVar) !== Type::VAR) {
if ($this->hasLocalVar($valueVar) && !$this->hasArgument($valueVar)) {
// Local declarations are emitted after the body is parsed, so a
// previously optimized scalar can still be promoted to Variant.
$this->context->localVars[$valueVar] = Type::VAR;
} else {
$this->fatalError($node, 'Cannot bind foreach reference to native variable of type ' . $this->getVarType($valueVar));
}
}
return $this->getIndent() . ' ' . $valueVar . ' = ' . $valueRefExpr . ';' . PHP_EOL;
return $this->getIndent() . ' ' . $valueRefExpr . '(' . $valueVar . ');' . PHP_EOL;
}
if ($this->isVarExpr($node->valueVar)) {
$this->checkVar($node, $valueVar);
if (!$this->hasVar($valueVar) || $this->getVarType($valueVar) !== Type::REF) {
$this->checkVar($node, $valueVar);
}
}
return $this->getIndent() . ' ' . $valueVar . ' = ' . $valueExpr . ';' . PHP_EOL;
}
@ -111,15 +119,18 @@ trait ForeachTrait
{
$iterator = $this->genTmpVarName();
$byRef = $node->byRef ? 'true' : 'false';
$code = "php::ForeachIterator $iterator{{$iterableVar}, $byRef};" . PHP_EOL;
$code .= "while ($iterator.next()) {" . PHP_EOL;
$scope = $this->class ? $this->getClassEntryPtr($this->getFullClassName()) : 'nullptr';
$code = '{' . PHP_EOL;
$this->indentLevel++;
$code .= $this->getIndent() . "php::ForeachIterator $iterator{{$iterableVar}, $byRef, $scope};" . PHP_EOL;
$code .= $this->getIndent() . "while ($iterator.next()) {" . PHP_EOL;
$this->indentLevel++;
$code .= $this->parseForeachKeyAssignment($node, $iterator . '.key()');
$code .= $this->parseForeachValueAssignment(
$node,
$iterator . '.value()',
$iterator . '.valueRef()',
$iterator . '.assignValueRef',
);
$body = $this->parseForeachBody($node);
@ -128,6 +139,8 @@ trait ForeachTrait
$code .= $this->parseBeforeStmtLines() . PHP_EOL;
$code .= $body . PHP_EOL;
$code .= $this->getIndent() . '}';
$this->indentLevel--;
$code .= PHP_EOL . $this->getIndent() . '}';
return $code;
}
@ -141,9 +154,6 @@ trait ForeachTrait
if ($type === Type::ARRAY) {
return $this->parseForeachIterable($node, $name);
} elseif ($type === Type::OBJECT) {
if ($node->byRef) {
$this->fatalError($node, 'Cannot use & with foreach');
}
return $this->parseForeachIterable($node, $name);
} elseif ($this->isStdContainerType($type)) {
return $this->parseForeachStdContainer($node);

@ -673,9 +673,8 @@ trait PropertyAccessTrait
$this->fatalError($var, 'Cannot use [] for array unset');
}
$array = $this->parseIdentifier($var->var);
if (($this->isStdMap($array) or $this->isStdOrderedMap($array))
and !empty($this->context->stdContainers[$array]['locking'])) {
$this->fatalError($var, 'Cannot delete element in std container in foreach loop');
if ($this->isStdVector($array) or $this->isStdMap($array) or $this->isStdOrderedMap($array)) {
$this->assertStdContainerStructureMutable($var, $array);
}
$dim = $this->parseIdentifier($var->dim);
if ($this->isStdContainer($array)) {

@ -20,6 +20,18 @@ use PhpParser\NodeAbstract;
trait StdContainerTrait
{
protected function isStdContainerIterating(string $var): bool
{
return !empty($this->context->stdContainers[$var]['iterationDepth']);
}
protected function assertStdContainerStructureMutable(NodeAbstract $node, string $var): void
{
if ($this->isStdContainerIterating($var)) {
$this->fatalError($node, "Cannot structurally modify std container `\${$var}` during foreach");
}
}
protected function isStdContainer(string $var): bool
{
return $this->hasLocalVar($var) and $this->isStdContainerType($this->getVarType($var));
@ -259,17 +271,20 @@ trait StdContainerTrait
}
$info = $this->getStdContainerInfo($left);
$container = $this->parseVariable($left->var);
if ($info['kind'] === 'vector' && $left->dim === null) {
if (!$this->isVarExpr($left->var)) {
$this->fatalError($left, 'std::vector append only supports a vector variable');
}
$vector = $this->parseVariable($left->var);
$this->assertStdContainerStructureMutable($left, $vector);
return $vector . '_ref.push_back(' . $this->convertStdValueExpr($info, $right) . ')';
}
if ($left->dim === null) {
$this->fatalError($left, 'std map expects a key');
}
$this->assertStdContainerStructureMutable($left, $container);
return $this->parseStdContainerOffsetSet($left, $this->convertStdValueExpr($info, $right));
}
@ -352,8 +367,10 @@ trait StdContainerTrait
protected function parseForeachStdContainer(Foreach_ $node): string
{
$container = $this->parseIdentifier($node->expr);
if ($this->isStdMap($container) or $this->isStdOrderedMap($container)) {
$this->context->stdContainers[$container]['locking'] = true;
$mutableContainer = !$this->isStdArray($container);
if ($mutableContainer) {
$this->context->stdContainers[$container]['iterationDepth'] =
($this->context->stdContainers[$container]['iterationDepth'] ?? 0) + 1;
}
$iterator = $this->genTmpVarName();
$code = "for (auto $iterator = {$container}_ref.begin(); $iterator != {$container}_ref.end(); ++$iterator) {" . PHP_EOL;
@ -385,7 +402,13 @@ trait StdContainerTrait
$code .= $this->getIndent() . "$valueVar = {$iterator}->second;" . PHP_EOL;
}
$body = $this->parseForeachBody($node);
try {
$body = $this->parseForeachBody($node);
} finally {
if ($mutableContainer) {
--$this->context->stdContainers[$container]['iterationDepth'];
}
}
$this->indentLevel--;
$code .= $this->parseBeforeStmtLines() . PHP_EOL;
@ -393,9 +416,6 @@ trait StdContainerTrait
$code .= $this->getIndent() . '}';
unset($this->context->objects[$valueVar]);
if ($this->isStdMap($container) or $this->isStdOrderedMap($container)) {
$this->context->stdContainers[$container]['locking'] = false;
}
return $code;
}

@ -0,0 +1,75 @@
--TEST--
foreach handles sparse mixed arrays, evaluates sources once, and releases cursors on control flow
--FILE--
<?php
final class ForeachArraySource
{
public static int $calls = 0;
public static function values(): array
{
++self::$calls;
return [0 => 'zero', 3 => 'three', 'name' => 'value'];
}
}
function stopEarly(array $values): string
{
foreach ($values as $value) {
return $value;
}
return 'empty';
}
function main(): void
{
$seen = [];
foreach (ForeachArraySource::values() as $key => $value) {
if ($key === 3) {
continue;
}
$seen[$key] = $value;
}
var_dump(ForeachArraySource::$calls, $seen);
$empty = [];
foreach ($empty as $value) {
echo "unreachable\n";
}
$inner = 1;
$references = [&$inner];
foreach ($references as $value) {
$value = 9;
}
var_dump($inner);
foreach ($references as &$value) {
$value = 11;
}
unset($value);
var_dump($inner);
var_dump(stopEarly(['first', 'second']));
try {
foreach (new ArrayIterator([1, 2]) as $value) {
throw new RuntimeException('stop');
}
} catch (RuntimeException $exception) {
echo $exception->getMessage(), "\n";
}
}
?>
--EXPECT--
int(1)
array(2) {
[0]=>
string(4) "zero"
["name"]=>
string(5) "value"
}
int(1)
int(11)
string(5) "first"
stop

@ -0,0 +1,112 @@
--TEST--
foreach invokes only required Iterator callbacks and cleans up exceptional cursors
--FILE--
<?php
class CallbackIterator implements Iterator
{
public static int $keyCalls = 0;
private int $position = 0;
public function __construct(private string $failure = '')
{
}
public function rewind(): void
{
if ($this->failure === 'rewind') throw new RuntimeException('rewind');
$this->position = 0;
}
public function valid(): bool
{
if ($this->failure === 'valid') throw new RuntimeException('valid');
return $this->position < 2;
}
public function current(): mixed
{
if ($this->failure === 'current') throw new RuntimeException('current');
return $this->position + 10;
}
public function key(): mixed
{
++self::$keyCalls;
if ($this->failure === 'key') throw new RuntimeException('key');
return $this->position;
}
public function next(): void
{
if ($this->failure === 'next') throw new RuntimeException('next');
++$this->position;
}
}
final class LifetimeIterator extends CallbackIterator
{
public static int $destroyed = 0;
public function __destruct()
{
++self::$destroyed;
}
}
final class LifetimeAggregate implements IteratorAggregate
{
public function getIterator(): Traversable
{
return new LifetimeIterator();
}
}
function consume(string $failure, bool $withKey): void
{
try {
if ($withKey) {
foreach (new CallbackIterator($failure) as $key => $value) {
}
} else {
foreach (new CallbackIterator($failure) as $value) {
}
}
} catch (RuntimeException $exception) {
echo $exception->getMessage(), "\n";
}
}
function main(): void
{
foreach (new CallbackIterator() as $value) {
echo $value, "\n";
}
var_dump(CallbackIterator::$keyCalls);
foreach (new CallbackIterator() as $key => $value) {
}
var_dump(CallbackIterator::$keyCalls);
consume('rewind', false);
consume('valid', false);
consume('current', false);
consume('next', false);
consume('key', true);
foreach (new LifetimeAggregate() as $value) {
break;
}
var_dump(LifetimeIterator::$destroyed);
}
?>
--EXPECT--
10
11
int(0)
int(2)
rewind
valid
current
next
key
int(1)

@ -0,0 +1,108 @@
--TEST--
foreach plain objects preserves scope, live properties, references, and typed-property rules
--FILE--
<?php
class ForeachScopeParent
{
public int $public = 1;
protected int $protected = 2;
private int $parentPrivate = 3;
}
class ForeachScopeChild extends ForeachScopeParent
{
private int $childPrivate = 4;
public int $typed = 5;
public readonly int $readonly;
public string $uninitialized;
public function __construct()
{
$this->readonly = 6;
}
public function visibleProperties(): array
{
$seen = [];
foreach ($this as $key => $value) {
$seen[$key] = $value;
}
return $seen;
}
}
class ForeachTypedOnly
{
public int $number = 7;
}
function main(): void
{
$object = (object) ['a' => 1, 'b' => 2];
$seen = [];
foreach ($object as $key => $value) {
$seen[$key] = $value;
if ($key === 'a') {
unset($object->b);
$object->c = 3;
}
}
var_dump($seen, $object->a, $object->c);
foreach ($object as &$value) {
$value *= 10;
}
unset($value);
var_dump($object->a, $object->c);
$scoped = new ForeachScopeChild();
var_dump($scoped->visibleProperties());
try {
foreach ($scoped as &$value) {
}
} catch (Error $error) {
var_dump(str_contains($error->getMessage(), 'readonly property'));
}
$typed = new ForeachTypedOnly();
try {
foreach ($typed as &$value) {
$value = 'invalid';
}
} catch (TypeError $error) {
var_dump(str_contains($error->getMessage(), 'int'));
}
var_dump($typed->number);
}
?>
--EXPECT--
array(2) {
["a"]=>
int(1)
["c"]=>
int(3)
}
int(1)
int(3)
int(10)
int(30)
array(6) {
["public"]=>
int(1)
["protected"]=>
int(2)
["childPrivate"]=>
int(4)
["typed"]=>
int(5)
["readonly"]=>
int(6)
["uninitialized"]=>
string(0) ""
}
bool(true)
bool(true)
int(7)

@ -0,0 +1,100 @@
--TEST--
foreach reference preserves COW, live mutations, and loop-variable aliases
--FILE--
<?php
function main(): void
{
$values = [1, 2];
$copy = $values;
foreach ($values as &$value) {
$value *= 10;
}
unset($value);
var_dump($values, $copy);
$values = [1, 2];
$seen = [];
foreach ($values as &$value) {
$seen[] = $value;
if ($value === 1) {
$values[] = 3;
}
}
unset($value);
var_dump($seen);
$values = [1, 2, 3];
$seen = [];
foreach ($values as $key => &$value) {
$seen[] = [$key, $value];
if ($key === 0) {
unset($values[1]);
}
}
unset($value);
var_dump($seen);
$value = 99;
foreach ($values as &$value) {
++$value;
}
unset($value);
foreach ([7, 8] as $value) {
echo $value, "\n";
}
$linked = [1, 2];
foreach ($linked as &$slot) {
}
foreach ([4, 5] as $slot) {
}
unset($slot);
var_dump($linked);
}
?>
--EXPECT--
array(2) {
[0]=>
int(10)
[1]=>
int(20)
}
array(2) {
[0]=>
int(1)
[1]=>
int(2)
}
array(3) {
[0]=>
int(1)
[1]=>
int(2)
[2]=>
int(3)
}
array(2) {
[0]=>
array(2) {
[0]=>
int(0)
[1]=>
int(1)
}
[1]=>
array(2) {
[0]=>
int(2)
[1]=>
int(3)
}
}
7
8
array(2) {
[0]=>
int(1)
[1]=>
int(5)
}

@ -0,0 +1,29 @@
--TEST--
std containers allow non-structural element updates during foreach
--FILE--
<?php
function main(): void
{
$vector = std::vector(Type::Int);
$vector[] = 1;
$vector[] = 2;
foreach ($vector as $vectorKey => $vectorValue) {
$vector[$vectorKey] += 10;
}
var_dump($vector[0], $vector[1]);
$map = std::ordered_map(Type::String, Type::Int);
$map['a'] = 3;
$map['b'] = 4;
foreach ($map as $mapKey => $mapValue) {
$map[$mapKey] += 20;
}
var_dump($map['a'], $map['b']);
}
?>
--EXPECT--
int(11)
int(12)
int(23)
int(24)
Loading…
Cancel
Save