Hermes-1
b89369cca1
- BUILD_PROXY now defaults to empty: the three GitHub fetches (llama.cpp zip x2, pinned CPython tarball) go straight to GitHub, so the Dockerfile builds as-is on unrestricted networks. Pass --build-arg BUILD_PROXY=socks5://<host>:<port> to route them through a SOCKS5 proxy instead. The hard-coded cr1.mse:2000 address is gone from every file. - socks2http.py + proxyrun.sh are now committed: the Dockerfile always shells out to proxyrun.sh (even in direct mode), so the bridge must be in the build context. - proxyrun.sh resolves socks2http.py relative to its own location (was hard-coded /src/socks2http.py), so the wrapper also works outside the image. - socks2http.py: --upstream defaults to empty (was the hard-coded address); empty upstream = plain TCP passthrough, still useful for testing.
159 lines
12 KiB
Docker
159 lines
12 KiB
Docker
# Strata, built FROM SOURCE (NVIDIA CUDA 13). The engine is compiled here, packaged as an engine component package and then
|
|
# installed into the same runtime image as the ready-made ones (docker/Dockerfile): same Terminal, same entrypoint, same /data volume.
|
|
#
|
|
# Most people want the ready-made images instead (nothing compiled): docs/DOCKER.md.
|
|
#
|
|
# docker build -t strata:src . # RTX 20/30/40/50 (compiles for each, ~25 min)
|
|
# docker build -t strata:src --build-arg CUDA_ARCHITECTURES=86 . # one card only (faster); 86 = RTX 30, 89 = RTX 40, 120 = RTX 50
|
|
# docker build -t strata:src-cuda12 --build-arg CUDA_IMAGE=nvidia/cuda:12.9.1-devel-ubuntu24.04 \
|
|
# --build-arg CUDA_ARCHITECTURES="60;61;70;75;80;86;89" --build-arg BUILD_VISION=0 . # GTX 10 / P100 / V100 (driver 575+)
|
|
#
|
|
# Dockerfile.v100 - V100 (sm_70) host. Also compiles the DeepSeek-V4.1 engine (strata-dsv41) into the engine
|
|
# package. The three GitHub fetches (llama.cpp source zip x2, the pinned CPython tarball x1) go straight to
|
|
# GitHub by default; pass --build-arg BUILD_PROXY=socks5://<host>:<port> to route them through a SOCKS5
|
|
# proxy (via the socks2http HTTP bridge, socks2http.py + proxyrun.sh at the build-context root). apt (distro
|
|
# mirror) and pip (PyPI) are NOT proxied - they are not on GitHub. If your build host has no internet at all
|
|
# (only the SOCKS), you would need to proxy those too (that requires bootstrapping python3 first; not done here).
|
|
# docker build -f Dockerfile.v100 -t strata:v100-dsv41 \
|
|
# --build-arg CUDA_IMAGE=nvidia/cuda:12.9.1-devel-ubuntu24.04 \
|
|
# --build-arg CUDA_ARCHITECTURES=70 --build-arg BUILD_DSV41=1 --build-arg BUILD_VISION=0 \
|
|
# --build-arg BUILD_ISA_FLOOR=none \
|
|
# --build-arg STRATA_COMMIT=$(git rev-parse HEAD) .
|
|
# # behind a SOCKS-only network, add:
|
|
# # --build-arg BUILD_PROXY=socks5://<host>:<port>
|
|
# BUILD_PROXY optional: the SOCKS5 endpoint the GitHub fetches go through; default "" = direct network access
|
|
# STRATA_COMMIT this checkout's commit. The build context has no .git (.dockerignore), so without it the
|
|
# terminal archive carries no COMMIT file and `strata bench run` on it is nonconforming.
|
|
# docker run --rm --gpus all -p 127.0.0.1:8080:8080 --ulimit memlock=-1 --shm-size 2g --stop-timeout 120 \
|
|
# -v strata-data:/data -e MODEL=IQ3_XXS -e API_KEY=change-me-please-123 strata:src
|
|
#
|
|
# docker build has no GPU, so the CUDA architectures are given here instead of read from nvidia-smi. The engine is a fat binary with
|
|
# a machine-code part per listed architecture (plus PTX for the newest, which later cards can JIT), and the runtime picks the one
|
|
# matching your card. Narrow CUDA_ARCHITECTURES to your card for a faster build. Native code for the CPU that builds the image is NOT
|
|
# used (STRATA_PORTABLE=ON), so the image runs on any x86-64 CPU with AVX2.
|
|
#
|
|
# Plain legacy-builder features only (no heredocs, no --mount). AMD (HIP) and Intel (SYCL) source builds: docs/AMD_HIP.md, docs/INTEL.md.
|
|
|
|
ARG CUDA_IMAGE=nvidia/cuda:13.0.0-devel-ubuntu24.04
|
|
ARG RUNTIME_BASE=ubuntu:24.04
|
|
|
|
# ---- 1. compile the engine and make its component package
|
|
FROM ${CUDA_IMAGE} AS engine-build
|
|
# The GitHub fetches below (the llama.cpp source zip) go straight to GitHub by default. BUILD_PROXY=socks5://...
|
|
# routes them through the socks2http bridge (socks2http.py + proxyrun.sh, both copied by `COPY . .`). Only GitHub
|
|
# is ever proxied; apt and pip hit the distro mirror / PyPI directly.
|
|
ARG BUILD_PROXY=
|
|
ENV DEBIAN_FRONTEND=noninteractive PYTHONUNBUFFERED=1 PATH=/venv/bin:$PATH
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
build-essential ca-certificates git libgomp1 libatomic1 python3 python3-pip python3-venv unzip \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
WORKDIR /src
|
|
COPY . .
|
|
ARG CUDA_ARCHITECTURES="75;80;86;89;120"
|
|
ARG BUILD_VISION=1
|
|
ARG BUILD_DSV41=0
|
|
# CPU instruction-set floor. "" (default) = STRATA_PORTABLE (AVX2 baseline, the normal release build).
|
|
# "none" = SSE4.2 floor, "avx" = AVX floor: for CPUs without AVX2 (e.g. a Pentium Gold G6605) the AVX2-baseline
|
|
# ggml-cpu the DeepSeek CPU experts run on traps with SIGILL (exit -4). Set to "none" to build for such a CPU.
|
|
ARG BUILD_ISA_FLOOR=
|
|
ARG ENGINE_VERSION=
|
|
RUN python3 -m venv /venv && /venv/bin/pip install --no-cache-dir -r requirements.txt -r requirements-build.txt
|
|
# llama.cpp at the pinned commit (ggml for the engine, mtmd for pictures), exactly as setup.py fetches it.
|
|
# GitHub fetch -> routed through the SOCKS5 proxy (see BUILD_PROXY above).
|
|
RUN sh /src/proxyrun.sh "$BUILD_PROXY" -- /venv/bin/python -c "import setup; print(setup.get_llama_cpp())"
|
|
RUN set -e; \
|
|
real=""; for a in $(echo "$CUDA_ARCHITECTURES" | tr ';,' ' '); do real="$real;${a}-real"; done; real=${real#;}; \
|
|
last=$(echo "$CUDA_ARCHITECTURES" | tr ';,' ' ' | awk '{print $NF}'); \
|
|
sm60=OFF; for a in $(echo "$CUDA_ARCHITECTURES" | tr ';,' ' '); do if [ "$a" -lt 75 ]; then sm60=ON; fi; done; \
|
|
/venv/bin/cmake -G Ninja -S . -B build -DCMAKE_BUILD_TYPE=Release -DSTRATA_ENABLE_CUDA=ON -DSTRATA_BUILD_TESTS=OFF \
|
|
-DSTRATA_EXPERIMENTAL_SM60=$sm60 \
|
|
-DSTRATA_BUILD_DSV41=$BUILD_DSV41 \
|
|
-DSTRATA_NATIVE_EXPERTS=ON -DSTRATA_Q6K_EXPERTS=ON -DSTRATA_MMQ_KQUANTS=ON \
|
|
-D$( [ -n "$BUILD_ISA_FLOOR" ] && echo "STRATA_ISA_FLOOR=$BUILD_ISA_FLOOR" || echo "STRATA_PORTABLE=ON" ) \
|
|
"-DCMAKE_CUDA_ARCHITECTURES=${real};${last}-virtual" -DSTRATA_GGML_DIR=/src/third_party/llama.cpp; \
|
|
tgt="strata strata_kq"; [ "$BUILD_DSV41" = "1" ] && tgt="$tgt strata-dsv41"; \
|
|
/venv/bin/cmake --build build --target $tgt -j "$(nproc)"; \
|
|
if [ "$BUILD_VISION" = "1" ]; then \
|
|
/venv/bin/cmake -G Ninja -S tools/vision -B build-vision -DCMAKE_BUILD_TYPE=Release -DLLAMA_DIR=/src/third_party/llama.cpp \
|
|
-DSTRATA_VISION_CUDA=ON -DGGML_CUDA_NCCL=OFF "-DCMAKE_CUDA_ARCHITECTURES=${real};${last}-virtual" \
|
|
-D$( [ -n "$BUILD_ISA_FLOOR" ] && echo "STRATA_ISA_FLOOR=$BUILD_ISA_FLOOR" || echo "STRATA_PORTABLE=ON" ); \
|
|
/venv/bin/cmake --build build-vision --target strata-vision -j "$(nproc)"; \
|
|
fi
|
|
RUN set -e; \
|
|
mkdir -p /stage /engine; \
|
|
cp build/strata /stage/; \
|
|
[ "$BUILD_DSV41" = "1" ] && cp build/strata-dsv41 /stage/ || true; \
|
|
[ -f build/libstrata_kq.so ] && cp build/libstrata_kq.so /stage/ || true; \
|
|
[ -f build-vision/bin/strata-vision ] && cp build-vision/bin/strata-vision /stage/ || true; \
|
|
ver="${ENGINE_VERSION:-$(sed -n 's/^[[:space:]]*project([^)]*VERSION[[:space:]]*\([0-9][0-9.]*\).*/\1/p' CMakeLists.txt | head -1)}"; \
|
|
cudafull=$(nvcc --version | sed -n 's/.*release \([0-9][0-9]*\.[0-9][0-9]*\).*/\1/p' | head -1); cudamajor=${cudafull%%.*}; \
|
|
if [ "$cudamajor" = 13 ]; then mindriver=580.65; else mindriver=575.51; fi; \
|
|
vision_archs=""; [ -f /stage/strata-vision ] && vision_archs=", \"vision_archs\": \"$CUDA_ARCHITECTURES\""; \
|
|
echo "{\"version\": \"$ver\", \"source\": \"docker build\", \"archs\": \"$CUDA_ARCHITECTURES\", \"cuda\": \"$cudafull\", \"portable\": true${vision_archs}}" > /stage/BUILD.json; \
|
|
python3 packaging/collect_libs.py --binary build/strata --search /usr/local/cuda/lib64 --out /stage --also 'libcublasLt.so*'; \
|
|
[ "$BUILD_DSV41" = "1" ] && python3 packaging/collect_libs.py --binary build/strata-dsv41 --search /usr/local/cuda/lib64 --out /stage --also 'libcublasLt.so*' || true; \
|
|
archs=""; for a in $(echo "$CUDA_ARCHITECTURES" | tr ';,' ' '); do archs="$archs --arch sm_$a"; done; \
|
|
last=$(echo "$CUDA_ARCHITECTURES" | tr ';,' ' ' | awk '{print $NF}'); \
|
|
python3 packaging/package_engine.py --src /stage --backend cuda --version "$ver" --out /engine $archs --arch "compute_$last" \
|
|
--min-driver nvidia=$mindriver --variant cuda$cudamajor --platform linux-x86_64 --build-id "docker-source-${ver}-cuda${cudafull}" --include SYSTEM-LIBS.txt --include 'libstrata_kq*' \
|
|
--include strata-vision $([ "$BUILD_DSV41" = "1" ] && echo --include strata-dsv41); \
|
|
for d in /engine/*/; do python3 packaging/package_engine.py --verify "$d"; python3 packaging/binary_deps.py "$d"; done
|
|
|
|
# ---- 2. the Terminal (the release archive, built here)
|
|
FROM ${RUNTIME_BASE} AS terminal
|
|
ARG STRATA_COMMIT=
|
|
# The build context has no .git (.dockerignore) and the image has no git: pass the host's commit with
|
|
# --build-arg STRATA_COMMIT=$(git rev-parse HEAD), so the terminal archive carries a COMMIT file
|
|
# (recorded by `strata bench run`; a standard benchmark without it is nonconforming).
|
|
ENV STRATA_BUILD_COMMIT=${STRATA_COMMIT}
|
|
# GitHub fetches in this stage (the pinned CPython tarball, the llama.cpp source zip) go straight to GitHub by
|
|
# default; BUILD_PROXY=socks5://... routes them through the socks2http bridge. apt (distro mirror) does not.
|
|
ARG BUILD_PROXY=
|
|
RUN apt-get update && apt-get install -y --no-install-recommends python3 ca-certificates git && rm -rf /var/lib/apt/lists/*
|
|
COPY . /src/
|
|
RUN sh /src/proxyrun.sh "$BUILD_PROXY" -- python3 /src/packaging/package_terminal.py --out /out --cache /cache \
|
|
&& mkdir -p /opt/strata \
|
|
&& tar -xzf /out/strata-terminal-*-linux-x86_64.tar.gz -C /opt/strata \
|
|
&& mv /opt/strata/strata-terminal-* /opt/strata/terminal \
|
|
&& /opt/strata/terminal/bin/strata --version
|
|
# llama.cpp at the commit setup.py pins: fetched here, so the first start does not depend on GitHub. Only gguf-py (the model preparation
|
|
# tools) and the file setup.py looks for (ggml/CMakeLists.txt) are kept: 170 MB become a few
|
|
RUN sh /src/proxyrun.sh "$BUILD_PROXY" -- sh -c 'cd /opt/strata/terminal && STRATA_CONTROL_INSTALL=1 STRATA_INSTALL_DIR=/opt/strata/seed PYTHONNOUSERSITE=1 /opt/strata/terminal/python/bin/python3 -c "import setup; print(setup.get_llama_cpp())"' \
|
|
&& rm -rf /opt/strata/seed/third_party/*.zip* \
|
|
&& cd /opt/strata/seed/third_party/llama.cpp \
|
|
&& find . -mindepth 1 -maxdepth 1 ! -name gguf-py ! -name ggml ! -name 'LICENSE*' -exec rm -rf {} + \
|
|
&& find ggml -mindepth 1 -depth ! -type d ! -name CMakeLists.txt -delete && find ggml -mindepth 1 -type d -empty -delete \
|
|
&& du -sh /opt/strata/seed
|
|
|
|
# ---- 3. the runtime image (the same as docker/Dockerfile's, FLAVOR=cuda)
|
|
FROM ${RUNTIME_BASE}
|
|
ARG STRATA_VERSION=dev
|
|
ENV DEBIAN_FRONTEND=noninteractive
|
|
RUN apt-get update && apt-get install -y --no-install-recommends tini ca-certificates libgomp1 libatomic1 libnuma1 zlib1g \
|
|
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/*
|
|
COPY --from=terminal /opt/strata/terminal /opt/strata/terminal
|
|
COPY --from=terminal /opt/strata/seed /opt/strata/seed
|
|
COPY --from=engine-build /engine/ /opt/strata/components/engine/
|
|
# The DeepSeek engine must be a REAL file inside its own package folder: setup.py (dsv41 setup, "its own
|
|
# package carries the libraries it needs") and launch/runner.py both put `exe.parent` on LD_LIBRARY_PATH.
|
|
# A symlink is fatal: the folder the symlink lives in has no libraries at all, so the engine dies with
|
|
# "libcudart.so.12: cannot open shared object file". Copying the whole package (~700 MB) is the cost of
|
|
# keeping the layout the launcher assumes.
|
|
RUN v=$(ls -d /opt/strata/components/engine/*/ | head -1) && \
|
|
mkdir -p /opt/strata/engine-dsv41 && cp -a "$v/." /opt/strata/engine-dsv41/ && \
|
|
{ [ -f /opt/strata/engine-dsv41/strata-dsv41 ] && chmod 0755 /opt/strata/engine-dsv41/strata-dsv41 || echo "no strata-dsv41 in the package (BUILD_DSV41=0?)"; }
|
|
COPY docker/entrypoint.sh docker/healthcheck.sh /opt/strata/
|
|
COPY docker/strata-wrapper.sh /usr/local/bin/strata
|
|
RUN chmod 0755 /opt/strata/entrypoint.sh /opt/strata/healthcheck.sh /usr/local/bin/strata
|
|
ENV STRATA_COMPONENTS_DIR=/opt/strata/components STRATA_INSTALL_DIR=/data/install XDG_STATE_HOME=/data/state \
|
|
XDG_CONFIG_HOME=/data/config HOME=/data/home PORT=8080 LANG=C.UTF-8 PYTHONUNBUFFERED=1 PYTHONNOUSERSITE=1 STRATA_IN_DOCKER=1 \
|
|
STRATA_DSV41_EXE=/opt/strata/engine-dsv41/strata-dsv41 \
|
|
NVIDIA_VISIBLE_DEVICES=all NVIDIA_DRIVER_CAPABILITIES=compute,utility
|
|
VOLUME ["/data"]
|
|
EXPOSE 8080
|
|
LABEL org.opencontainers.image.title="Strata" com.strata.flavor="cuda" com.strata.build="from-source" org.opencontainers.image.version="${STRATA_VERSION}"
|
|
HEALTHCHECK --interval=30s --timeout=8s --start-period=2h --retries=3 CMD ["/opt/strata/healthcheck.sh"]
|
|
STOPSIGNAL SIGTERM
|
|
ENTRYPOINT ["/usr/bin/tini", "--", "/opt/strata/entrypoint.sh"]
|
|
CMD ["run"]
|