fix(scanner): follow symlinked source directories (#116) --skip-tests

Composer path repositories install a package as a symlink, so a source
directory reached through one contributed no files at all: PHP's
RecursiveDirectoryIterator does not descend into symlinked directories
unless asked, because RecursiveIteratorIterator calls hasChildren() and
its $allowLinks argument defaults to false.

The build still reported success, and the missing classes only surfaced
at runtime as `class 'X' is undefined`.

Scan with FilesystemIterator::FOLLOW_SYMLINKS. A linked directory is an
ordinary source entry with no setting of its own; what is compiled stays
with `sources` and `ignore`.

Exclusions match the path that reached a file, the one the project wrote
and the scanner traversed. An `ignore` entry is no longer resolved to its
link target, which compared a real path against a linked one and so never
matched.

Real paths are kept for identity alone. A link is refused when it closes
the branch that reached it, so a link to one of its own ancestors cannot
recurse; files reachable through several links are reduced to one path
after exclusions have run, since deduplicating before that would drop an
allowed alias along with an excluded one.

README-CN.md

@@ -352,7 +352,9 @@ embedded-files:
 
 路径以最外层项目 YAML 文件所在目录为基准。source 可以是文件或目录；条件 source 支持
 `PHP_VERSION`、`PHP_VERSION_ID` 和 `PHP_OS_FAMILY`。命令行参数优先于 YAML
-中的同名配置。原生链接依赖应写入 `link-libs`；`ext-deps` 会生成
+中的同名配置。扫描源码目录时会进入符号链接指向的目录，因此通过 Composer path
+仓库安装的依赖（以符号链接方式安装）会像其他源码一样被编译；如需排除，请在
+`ignore` 中按访问该目录所用的路径书写。原生链接依赖应写入 `link-libs`；`ext-deps` 会生成
 `ZEND_MOD_REQUIRED`，缺少所需 PHP 扩展时由 Zend 拒绝加载模块。
 `version` 用于设置 Zend 模块版本。`info` 映射可配置任意标签和值，并显示在模块
 独立的 `phpinfo()` 区块中。

README.md

@@ -386,7 +386,10 @@ section. Relative project paths are resolved against the outermost project file.
 
 Paths are resolved relative to the outermost project YAML file. A source entry may be a file or
 directory; conditional entries support `PHP_VERSION`, `PHP_VERSION_ID`, and
-`PHP_OS_FAMILY`. CLI arguments override their YAML counterparts. Native linker
+`PHP_OS_FAMILY`. CLI arguments override their YAML counterparts. Scanning a
+source directory descends into symlinked directories, so a dependency installed
+by a Composer path repository -- which is a symlink -- is compiled like any
+other source; `ignore` excludes it, written as the path that reaches it. Native linker
 dependencies belong in `link-libs`; `ext-deps` writes `ZEND_MOD_REQUIRED`
 entries so Zend can reject loading when a required PHP extension is missing.
 `version` provides the Zend module version. The `info` mapping accepts arbitrary

docs/en/COMPILER_CLI.md

@@ -117,6 +117,21 @@ Corresponding long options:
 
 When a `project.yml` is passed, command-line arguments take precedence over same-named settings in the YAML. For the project file format, see the user documentation and the project configuration parser in the code.
 
+### Symlinked source directories
+
+Scanning a source directory descends into symlinked directories, so a dependency
+installed by a Composer path repository -- which is installed as a symlink -- is
+compiled like any other source. A link pointing at one of its own ancestors does
+not recurse, and a file reached through more than one link is compiled once.
+
+Excluding one is the ordinary `ignore` entry, written as the path that reaches
+it rather than the path it points at:
+
+```yaml
+ignore:
+  - vendor/vendor/mylib
+```
+
 ### Precompiled object files
 
 A project can add object files produced by an external native toolchain as

docs/zh-cn/COMPILER_CLI.md

@@ -117,6 +117,20 @@ TypePHP 和 PHPX 的最低运行时版本均为 PHP 8.4。`--php-version` 与实
 
 传入 `project.yml` 时，命令行参数优先于 YAML 中的同名配置。项目文件格式参见用户文档及代码中的项目配置解析器。
 
+### 符号链接的源码目录
+
+扫描源码目录时会进入符号链接指向的目录，因此通过 Composer path 仓库安装的依赖
+（以符号链接方式安装）会像其他源码一样被编译。指向自身上级目录的链接不会无限
+递归；通过多个链接都能访问到的文件只会被编译一次。
+
+如需排除，使用常规的 `ignore` 配置，并按访问该目录所用的路径书写，而不是链接
+指向的目标路径：
+
+```yaml
+ignore:
+  - vendor/vendor/mylib
+```
+
 ### 预编译对象文件
 
 项目可以把外部工具链生成的对象文件作为通用链接输入：

phpunit/src/Build/FileScannerTest.php

@@ -0,0 +1,175 @@
+<?php
+
+namespace TypePhpTest\Build;
+
+use PHPUnit\Framework\TestCase;
+use TypePhp\Build\FileScanner;
+use TypePhp\CompilerTest;
+
+final class FileScannerTest extends TestCase
+{
+    private string $root;
+
+    protected function setUp(): void
+    {
+        $this->root = realpath(sys_get_temp_dir()) . '/typephp-file-scanner-' . bin2hex(random_bytes(6));
+        mkdir($this->root . '/project/src', 0777, true);
+        mkdir($this->root . '/outside/src', 0777, true);
+        file_put_contents($this->root . '/project/src/Own.php', "<?php\nclass Own {}\n");
+        file_put_contents($this->root . '/outside/src/Linked.php', "<?php\nclass Linked {}\n");
+    }
+
+    protected function tearDown(): void
+    {
+        $this->removeDirectory($this->root);
+    }
+
+    /**
+     * A Composer path repository installs a package as a symlink, so the files
+     * behind one have to be scanned like any other source.
+     */
+    public function testFollowsSymlinkedDirectory(): void
+    {
+        symlink($this->root . '/outside', $this->root . '/project/vendor-link');
+
+        $files = (new FileScanner($this->root . '/project'))->scan();
+
+        $this->assertSame([
+            $this->root . '/project/src/Own.php',
+            $this->root . '/project/vendor-link/src/Linked.php',
+        ], $files);
+    }
+
+    public function testFollowsSymlinkedFile(): void
+    {
+        symlink($this->root . '/outside/src/Linked.php', $this->root . '/project/src/Linked.php');
+
+        $files = (new FileScanner($this->root . '/project'))->scan();
+
+        $this->assertSame([
+            $this->root . '/project/src/Linked.php',
+            $this->root . '/project/src/Own.php',
+        ], $files);
+    }
+
+    /**
+     * A link pointing at one of its own ancestors must not recurse forever.
+     */
+    public function testSymlinkCycleIsScannedOnce(): void
+    {
+        symlink($this->root . '/project', $this->root . '/project/src/loop');
+
+        $files = (new FileScanner($this->root . '/project'))->scan();
+
+        $this->assertSame([$this->root . '/project/src/Own.php'], $files);
+    }
+
+    /**
+     * Two links to one directory are one source file seen twice, and compiling
+     * it twice would define its symbols twice.
+     */
+    public function testAliasesOfOneFileAreScannedOnce(): void
+    {
+        symlink($this->root . '/outside', $this->root . '/project/link-a');
+        symlink($this->root . '/outside', $this->root . '/project/link-b');
+
+        $files = (new FileScanner($this->root . '/project'))->scan();
+
+        $this->assertSame([
+            $this->root . '/project/link-a/src/Linked.php',
+            $this->root . '/project/src/Own.php',
+        ], $files);
+    }
+
+    /**
+     * Excluding one alias says nothing about the other. Deduplicating before
+     * exclusions would drop the allowed path along with the excluded one.
+     */
+    public function testExcludingOneAliasKeepsTheOther(): void
+    {
+        symlink($this->root . '/outside', $this->root . '/project/link-a');
+        symlink($this->root . '/outside', $this->root . '/project/link-b');
+
+        foreach (['link-a' => 'link-b', 'link-b' => 'link-a'] as $excluded => $kept) {
+            $files = (new FileScanner($this->root . '/project'))
+                ->addExcludePattern($this->root . '/project/' . $excluded . '/src/*')
+                ->scan();
+
+            $this->assertSame([
+                $this->root . '/project/' . $kept . '/src/Linked.php',
+                $this->root . '/project/src/Own.php',
+            ], $files, "excluding {$excluded} must not hide {$kept}");
+        }
+    }
+
+    /**
+     * `ignore` names the path that reaches a directory, not the path it points
+     * at, so the scanned path is what an entry has to be compared against.
+     */
+    public function testYamlIgnoreExcludesLinkedDirectory(): void
+    {
+        symlink($this->root . '/outside', $this->root . '/project/vendor-link');
+        file_put_contents(
+            $this->root . '/project/project.yml',
+            "name: demo\nsources:\n  - .\nignore:\n  - vendor-link\n",
+        );
+
+        $this->assertSame(
+            [$this->root . '/project/src/Own.php'],
+            $this->scanProject($this->root . '/project/project.yml'),
+        );
+    }
+
+    /**
+     * An entry that names the link target leaves the linked path compiled: it
+     * describes a directory the scan never reached.
+     */
+    public function testYamlIgnoreOfTheLinkTargetLeavesTheLinkedPath(): void
+    {
+        symlink($this->root . '/outside', $this->root . '/project/vendor-link');
+        file_put_contents(
+            $this->root . '/project/project.yml',
+            "name: demo\nsources:\n  - .\nignore:\n  - ../outside\n",
+        );
+
+        $this->assertSame([
+            $this->root . '/project/src/Own.php',
+            $this->root . '/project/vendor-link/src/Linked.php',
+        ], $this->scanProject($this->root . '/project/project.yml'));
+    }
+
+    /** @return list<string> */
+    private function scanProject(string $projectFile): array
+    {
+        $compiler   = CompilerTest::create(dirname($projectFile));
+        $reflection = new \ReflectionClass($compiler);
+
+        $parse  = $reflection->getMethod('parseProjectYaml');
+        $filter = $reflection->getMethod('filterIgnoredFiles');
+
+        return array_values($filter->invoke($compiler, $parse->invoke($compiler, $projectFile)));
+    }
+
+    private function removeDirectory(string $directory): void
+    {
+        if (!is_dir($directory)) {
+            return;
+        }
+
+        foreach (scandir($directory) as $entry) {
+            if ($entry === '.' || $entry === '..') {
+                continue;
+            }
+
+            $path = $directory . '/' . $entry;
+            if (is_link($path) || !is_dir($path)) {
+                unlink($path);
+                continue;
+            }
+
+            $this->removeDirectory($path);
+        }
+
+        rmdir($directory);
+    }
+}

src/Build/FileScanner.php

@@ -84,9 +84,7 @@ class FileScanner
     public function scan(): array
     {
         $files    = [];
-        $iterator = new \RecursiveIteratorIterator(
+        $iterator = new \RecursiveIteratorIterator($this->createDirectoryIterator());
-            new \RecursiveDirectoryIterator($this->directory, \FilesystemIterator::SKIP_DOTS)
-        );
 
         foreach ($iterator as $file) {
             if ($file->isFile()) {
@@ -106,7 +104,94 @@ class FileScanner
         // keep both generated code and cache classification deterministic.
         sort($files, SORT_STRING);
 
-        return $files;
+        return $this->deduplicateByRealPath($files);
+    }
+
+    /**
+     * Descend into symlinked directories.
+     *
+     * RecursiveDirectoryIterator does not follow them unless asked to:
+     * RecursiveIteratorIterator calls hasChildren(), whose $allowLinks argument
+     * defaults to false. Composer path repositories install a package as a
+     * symlink, so without FOLLOW_SYMLINKS a source directory that lives behind
+     * one contributes no files at all.
+     */
+    private function createDirectoryIterator(): \RecursiveIterator
+    {
+        $iterator = new \RecursiveDirectoryIterator(
+            $this->directory,
+            \FilesystemIterator::SKIP_DOTS | \FilesystemIterator::FOLLOW_SYMLINKS,
+        );
+
+        return new \RecursiveCallbackFilterIterator(
+            $iterator,
+            fn(\SplFileInfo $entry): bool => !$this->closesLoop($entry),
+        );
+    }
+
+    /**
+     * Whether descending into an entry would repeat the branch that reached it,
+     * which is what a link pointing at one of its own ancestors does. Only a
+     * link can: every other directory resolves below the one holding it.
+     *
+     * The branch is the whole test, never a set of everything visited so far.
+     * Two links to one directory are two ordinary source paths, and either of
+     * them may be the one a project excludes, so which of the two is the same
+     * file is decided after exclusions instead, by real path.
+     */
+    private function closesLoop(\SplFileInfo $entry): bool
+    {
+        if (!$entry->isLink() || !$entry->isDir()) {
+            return false;
+        }
+
+        $target = $entry->getRealPath();
+        if ($target === false) {
+            return false;
+        }
+
+        $ancestor = dirname($entry->getPathname());
+        while (true) {
+            if (realpath($ancestor) === $target) {
+                return true;
+            }
+            if ($ancestor === $this->directory) {
+                return false;
+            }
+            $parent = dirname($ancestor);
+            if ($parent === $ancestor) {
+                return false;
+            }
+            $ancestor = $parent;
+        }
+    }
+
+    /**
+     * One file reachable through several links is still one source file, and
+     * compiling it twice would define its symbols twice.
+     *
+     * Exclusions have already run, so a path the project excluded can never be
+     * the alias that survives here. The list is sorted, so which alias survives
+     * does not depend on directory-entry order.
+     *
+     * @param list<string> $files
+     * @return list<string>
+     */
+    private function deduplicateByRealPath(array $files): array
+    {
+        $unique = [];
+        $seen   = [];
+
+        foreach ($files as $file) {
+            $identity = realpath($file) ?: $file;
+            if (isset($seen[$identity])) {
+                continue;
+            }
+            $seen[$identity] = true;
+            $unique[]        = $file;
+        }
+
+        return $unique;
     }
 
     private function isExcluded(string $filePath): bool

src/Translator.php

@@ -3898,6 +3898,45 @@ CODE;
         return $baseDir . '/' . $path;
     }
 
+    /**
+     * Resolve `.` and `..` without touching the filesystem.
+     *
+     * realpath() would do this too, but it also follows every symlink on the
+     * way, which is exactly what a path compared against a scanned one must
+     * not do.
+     */
+    protected function normalizeLexicalPath(string $path): string
+    {
+        $prefix = '';
+        if (preg_match('/^[A-Za-z]:/', $path) === 1) {
+            $prefix = substr($path, 0, 2);
+            $path   = substr($path, 2);
+        }
+        if (DIRECTORY_SEPARATOR === '\\') {
+            $path = str_replace('\\', '/', $path);
+        }
+
+        $absolute = str_starts_with($path, '/');
+        $segments = [];
+        foreach (explode('/', $path) as $segment) {
+            if ($segment === '' || $segment === '.') {
+                continue;
+            }
+            if ($segment === '..' && $segments !== [] && end($segments) !== '..') {
+                array_pop($segments);
+                continue;
+            }
+            $segments[] = $segment;
+        }
+
+        $normalized = ($absolute ? '/' : '') . implode('/', $segments);
+        if ($prefix !== '') {
+            $normalized = $prefix . $normalized;
+        }
+
+        return str_replace('/', DIRECTORY_SEPARATOR, $normalized);
+    }
+
     protected function isAbsolutePath(string $path): bool
     {
         return $path !== ''
@@ -4256,14 +4295,25 @@ CODE;
                 $this->error('`ignore` must be array');
             }
             foreach ($ignore as $src) {
-                $realPath = $this->getAbsolutePath($src, $projectDir);
+                $path = $this->normalizeLexicalPath($this->resolvePath($src, $projectDir, 'Ignore path'));
-                if (!$realPath) {
+                if (!file_exists($path)) {
                     // Ignore entries describe optional exclusions. Projects often
                     // share one configuration across dependency versions where an
                     // excluded file or directory may not exist.
                     continue;
                 }
-                $this->ignorePaths[] = $realPath;
+                // The scanner reports the path it traversed, symlinks and all,
+                // so an entry naming a linked directory has to be compared as
+                // the project wrote it. Resolving it first would compare a real
+                // path against a linked one and never match.
+                $this->ignorePaths[] = $path;
+                // A source entry is resolved before it is scanned, so a project
+                // that links its source root traverses real paths instead. Keep
+                // the target as well, for that case.
+                $realPath = realpath($path);
+                if ($realPath !== false && $realPath !== $path) {
+                    $this->ignorePaths[] = $realPath;
+                }
             }
         }