Skip to content

Commit 9a59417

Browse files
TysonAndreTysonAndre
authored
[skip ci] Document how to quickly check if jit .dasc files transpile, how to test the jit in different architectures. (#7768)
1 parent a5f7821 commit 9a59417

File tree

3 files changed

+131
-2
lines changed

3 files changed

+131
-2
lines changed

‎ext/opcache/jit/Dockerfile.arm64.example

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Force this to build with arm64 even when the host architecture is different.
2+
# This requires that cross-compilation support be enabled with the steps in https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/
3+
FROM --platform=arm64 ubuntu:20.04
4+
RUN apt-get update -y
5+
# DEBIAN_FRONTEND=noninteractive is needed to stop the tzdata installation from hanging.
6+
ENV DEBIAN_FRONTEND=noninteractive
7+
RUN apt-get install -y tzdata
8+
RUN apt-get install -y pkg-config build-essential autoconf bison re2c \
9+
libxml2-dev libsqlite3-dev
10+
11+
ADD . /php-src/
12+
WORKDIR /php-src
13+
RUN ./buildconf
14+
# Compile a minimal debug build. --enable-debug adds runtime assertions and is slower than regular builds.
15+
RUN ./configure --enable-debug --disable-all --enable-opcache && make clean && make -j$(nproc)

‎ext/opcache/jit/README.md

Lines changed: 115 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,5 +24,119 @@ was almost 100 times slower, making it prohibitively expensive to use.
2424
[The unofficial DynASM Documentation](https://corsix.github.io/dynasm-doc/tutorial.html)
2525
has a tutorial, reference, and instruction listing.
2626

27-
`zend_jit_x86.dasc` gets automatically converted to `zend_jit_x86.c` by the bundled
27+
In x86 builds, `zend_jit_x86.dasc` gets automatically converted to `zend_jit_x86.c` by the bundled
2828
`dynasm` during `make`.
29+
30+
In arm64 builds, `zend_jit_arm64.dasc` gets automatically converted to `zend_jit_arm64.c` by the bundled
31+
`dynasm` during `make`.
32+
33+
Running tests of the JIT
34+
------------------------
35+
36+
Then, to test the JIT, e.g. with opcache.jit=tracing, an example command
37+
based on what is used to test in Azure CI:
38+
39+
```
40+
make test TESTS="-d opcache.jit_buffer_size=16M -d opcache.enable=1 -d opcache.enable_cli=1 -d opcache.protect_memory=1 -d opcache.jit=tracing --repeat 2 --show-diff -j$(nproc) ext/opcache Zend"
41+
```
42+
43+
- `opcache.jit_buffer_size=16M` enables the JIT in tests by providing 16 megabytes of
44+
memory to use with the JIT to test with.
45+
- `opcache.protect_memory=1` will detect writing to memory that is meant to be
46+
read-only, which is sometimes the cause of opcache bugs.
47+
- `--repeat 2` is optional, but used in CI since some JIT bugs only show up after processing a
48+
request multiple times (the first request compiles the trace and the second executes it)
49+
- `-j$(nproc)` runs as many workers to run tests as there are CPUs.
50+
- `ext/opcache/` and `Zend` are the folders with the tests to run, in this case opcache
51+
and the Zend engine itself. If no folders are provided, all tests are run.
52+
53+
When investigating test failures such as segmentation faults,
54+
configuring the build of php with `--enable-address-sanitizer` to enable
55+
[AddressSanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizer) is often useful.
56+
57+
Some of the time, adding `-m --show-mem` to the `TESTS` configuration is also useful to test with [valgrind](https://valgrind.org/) to detect out of bounds memory accesses.
58+
Using valgrind is slower at detecting invalid memory read/writes than AddressSanitizer when running large numbers of tests, but does not require rebuilding php.
59+
60+
Note that the JIT supports 3 different architectures: `X86_64`, `i386`, and `arm64`.
61+
62+
Miscellaneous
63+
-------------
64+
65+
### Checking dasc files for in a different architecture
66+
67+
The following command can be run to manually check if the modified `.dasc code` is at least transpilable
68+
for an architecture you're not using, e.g.:
69+
70+
For arm64: `ext/opcache/minilua ext/opcache/jit/dynasm/dynasm.lua -D ARM64=1 -o ext/opcache/jit/zend_jit_arm64.ignored.c ext/opcache/jit/zend_jit_arm64.dasc`
71+
72+
For x86_64: `ext/opcache/minilua ext/opcache/jit/dynasm/dynasm.lua -D X64=1 -o ext/opcache/jit/zend_jit_x86.ignored.c ext/opcache/jit/zend_jit_x86.dasc`
73+
74+
For i386 (i.e. 32-bit): `ext/opcache/minilua ext/opcache/jit/dynasm/dynasm.lua -o ext/opcache/jit/zend_jit_x86.ignored.c ext/opcache/jit/zend_jit_x86.dasc`
75+
76+
### How to build 32-bit builds on x86_64 environments
77+
78+
Refer to [../../../azure/i386](../../../azure/i386/apt.yml) for examples of
79+
dependencies to install.
80+
81+
If you are running this natively (outside of Docker or a VM):
82+
83+
- Consider running in docker/a VM instead if you are unfamiliar with this.
84+
- Avoid purging packages.
85+
- Avoid `-y` - if the package manager warns you that the dependencies conflict
86+
then **don't** try to force install them.
87+
88+
#### Prerequisites for 32-bit builds
89+
90+
This assumes you are using a Debian-based Linux distribution and have already
91+
set up prerequisites for regular development.
92+
93+
```
94+
sudo dpkg --add-architecture i386
95+
sudo apt-get update -y
96+
# As well as anything else from azure/i386/apt.yml that you're testing locally
97+
sudo apt-get install \
98+
gcc-multilib g++-multilib \
99+
libxml2-dev:i386 \
100+
libc6:i386
101+
```
102+
103+
#### Compiling 32-bit builds
104+
105+
This assumes you are using a Debian-based Linux distribution and have already
106+
set up prerequisites for 32-bit development.
107+
108+
```
109+
export LDFLAGS=-L/usr/lib/i386-linux-gnu
110+
export CFLAGS='-m32'
111+
export CXXFLAGS='-m32'
112+
export PKG_CONFIG=/usr/bin/i686-linux-gnu-pkg-config
113+
./configure --disable-all --enable-opcache --build=i686-pc-linux-gnu
114+
make -j$(nproc)
115+
```
116+
117+
#### Running tests of the JIT on 32-bit builds
118+
119+
See the section "Running tests of the JIT".
120+
121+
### Testing the jit with arm64 on x86 computers
122+
123+
https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/
124+
may be useful for local development.
125+
126+
Note that this is slower than compiling and testing natively.
127+
128+
```
129+
# After following steps in https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/
130+
cp .gitignore .dockerignore
131+
echo .git >> .dockerignore
132+
133+
docker build --network=host -t php-src-arm64-example -f ext/opcache/jit/Dockerfile.arm64.example .
134+
docker run -it --rm php-src-arm64-example
135+
```
136+
137+
Then, the docker image can be used to run tests with `make test`.
138+
For example, to test `ext/opcache` in parallel with the tracing JIT enabled:
139+
140+
```
141+
docker run -it php-src-arms-example make test TESTS="-d opcache.jit_buffer_size=16M -d opcache.enable=1 -d opcache.enable_cli=1 -d opcache.protect_memory=1 -d opcache.jit=tracing --repeat 2 --show-diff -j$(nproc) ext/opcache"
142+
```

‎sapi/fuzzer/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
Fuzzing SAPI for PHP
22
--------------------
33

4-
The following `./configure` options can be used to enable the fuzzing SAPI, as well as all availablefuzzers. If you don't build the exif/json/mbstring extensions, fuzzers for these extensions will not be built.
4+
The following `./configure` options can be used to enable the fuzzing SAPI, as well as all available fuzzers. If you don't build the exif/json/mbstring extensions, fuzzers for these extensions will not be built.
55

66
```sh
77
CC=clang CXX=clang++ \

0 commit comments

Comments
 (0)