build: add frontend-builder image and bake buf into java-builder
ci/frontend-builder (new): Node 24 + pnpm 10 + buf 1.55 + global @bufbuild/protoc-gen-es 2.4 + Playwright chromium runtime libs + ORAS. Used by KollectAI-ETL frontend CI to run TS proto codegen locally (buf.gen.yaml uses `local: protoc-gen-es`) instead of the rate-limited BSR remote-plugin path. ci/java-builder: added the standalone buf binary so `buf lint` runs in CI without an inline curl install. Backend Java codegen stays in the Maven build (protobuf-maven-plugin), so no protoc plugins are added to this image. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,91 @@
|
|||||||
|
# KollectAI CI - Frontend Builder Image
|
||||||
|
#
|
||||||
|
# Pre-baked build environment for SvelteKit frontend CI jobs and
|
||||||
|
# protobuf TypeScript code generation.
|
||||||
|
# Contains: Node.js, pnpm, buf CLI, @bufbuild/protoc-gen-es plugin
|
||||||
|
# (global so buf 'local: protoc-gen-es' resolves via PATH), Playwright
|
||||||
|
# system deps, ORAS CLI, common build tools.
|
||||||
|
#
|
||||||
|
# Build:
|
||||||
|
# docker build -t 192.168.1.72/kollect-tools/ci/frontend-builder:latest ci/frontend-builder/
|
||||||
|
#
|
||||||
|
# Usage in CI:
|
||||||
|
# container:
|
||||||
|
# image: 192.168.1.72/kollect-tools/ci/frontend-builder:latest
|
||||||
|
|
||||||
|
ARG NODE_MAJOR=24
|
||||||
|
FROM node:${NODE_MAJOR}-bookworm-slim
|
||||||
|
|
||||||
|
ARG PNPM_VERSION=10.15.0
|
||||||
|
ARG BUF_VERSION=1.55.0
|
||||||
|
# Pin protoc-gen-es to a known-good major. Bump in lockstep with the
|
||||||
|
# frontend's @bufbuild/protobuf runtime version in package.json.
|
||||||
|
ARG PROTOC_GEN_ES_VERSION=2.4.0
|
||||||
|
ARG ORAS_VERSION=1.2.2
|
||||||
|
|
||||||
|
# ─────────────────────────────────────────────────────────────────────
|
||||||
|
# System dependencies
|
||||||
|
#
|
||||||
|
# The Playwright system libs (libnss3 ... libasound2) are pre-installed
|
||||||
|
# so test-e2e jobs can skip `playwright install --with-deps` and just
|
||||||
|
# run `playwright install chromium` (browser binary still cached at
|
||||||
|
# ~/.cache/ms-playwright in CI). Saves ~30s per E2E run.
|
||||||
|
# ─────────────────────────────────────────────────────────────────────
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
ca-certificates \
|
||||||
|
curl \
|
||||||
|
git \
|
||||||
|
jq \
|
||||||
|
unzip \
|
||||||
|
# Playwright chromium runtime libs
|
||||||
|
libasound2 \
|
||||||
|
libatk1.0-0 \
|
||||||
|
libatk-bridge2.0-0 \
|
||||||
|
libcups2 \
|
||||||
|
libdbus-1-3 \
|
||||||
|
libdrm2 \
|
||||||
|
libgbm1 \
|
||||||
|
libnspr4 \
|
||||||
|
libnss3 \
|
||||||
|
libxcomposite1 \
|
||||||
|
libxdamage1 \
|
||||||
|
libxfixes3 \
|
||||||
|
libxkbcommon0 \
|
||||||
|
libxrandr2 \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# ─────────────────────────────────────────────────────────────────────
|
||||||
|
# pnpm via corepack (ships with Node.js)
|
||||||
|
# ─────────────────────────────────────────────────────────────────────
|
||||||
|
RUN corepack enable \
|
||||||
|
&& corepack prepare "pnpm@${PNPM_VERSION}" --activate
|
||||||
|
|
||||||
|
# ─────────────────────────────────────────────────────────────────────
|
||||||
|
# buf CLI - single static binary, used for `buf lint` and `buf generate`
|
||||||
|
# ─────────────────────────────────────────────────────────────────────
|
||||||
|
RUN curl -fsSL "https://github.com/bufbuild/buf/releases/download/v${BUF_VERSION}/buf-Linux-x86_64" \
|
||||||
|
-o /usr/local/bin/buf \
|
||||||
|
&& chmod +x /usr/local/bin/buf
|
||||||
|
|
||||||
|
# ─────────────────────────────────────────────────────────────────────
|
||||||
|
# protoc-gen-es - TypeScript codegen plugin for buf 'local:' references.
|
||||||
|
# Installed globally so it lands on PATH; buf v2 resolves
|
||||||
|
# `local: protoc-gen-es` via PATH lookup.
|
||||||
|
# ─────────────────────────────────────────────────────────────────────
|
||||||
|
RUN npm install -g "@bufbuild/protoc-gen-es@${PROTOC_GEN_ES_VERSION}"
|
||||||
|
|
||||||
|
# ─────────────────────────────────────────────────────────────────────
|
||||||
|
# ORAS CLI - for uploading artifacts (test reports, audit logs) to Harbor
|
||||||
|
# ─────────────────────────────────────────────────────────────────────
|
||||||
|
RUN curl -fsSL "https://github.com/oras-project/oras/releases/download/v${ORAS_VERSION}/oras_${ORAS_VERSION}_linux_amd64.tar.gz" \
|
||||||
|
| tar -xz -C /usr/local/bin oras
|
||||||
|
|
||||||
|
WORKDIR /workspace
|
||||||
|
|
||||||
|
# Verify installation
|
||||||
|
RUN node --version \
|
||||||
|
&& pnpm --version \
|
||||||
|
&& buf --version \
|
||||||
|
&& protoc-gen-es --version \
|
||||||
|
&& oras version \
|
||||||
|
&& jq --version
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
# Frontend Builder — CI Image
|
||||||
|
|
||||||
|
Pre-baked build environment for KollectAI-ETL frontend (SvelteKit) CI jobs and protobuf TypeScript code generation.
|
||||||
|
|
||||||
|
## What's included
|
||||||
|
|
||||||
|
- Node.js 24 (`node:24-bookworm-slim`)
|
||||||
|
- pnpm 10 (via corepack)
|
||||||
|
- [buf](https://buf.build) CLI — for `buf lint` and `buf generate`
|
||||||
|
- [`@bufbuild/protoc-gen-es`](https://www.npmjs.com/package/@bufbuild/protoc-gen-es) — installed globally so `buf` v2 `local: protoc-gen-es` resolves via PATH (no BSR remote-plugin calls)
|
||||||
|
- Playwright chromium runtime libraries pre-installed (`libnss3`, `libgbm1`, `libasound2`, etc.) — so `playwright install chromium` works without `--with-deps` apt fetches
|
||||||
|
- ORAS CLI — for uploading test reports / audit artifacts to Harbor
|
||||||
|
- `git`, `jq`, `curl`, `unzip`
|
||||||
|
|
||||||
|
## Build
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker build -t 192.168.1.72/kollect-tools/ci/frontend-builder:latest ci/frontend-builder/
|
||||||
|
docker push 192.168.1.72/kollect-tools/ci/frontend-builder:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
### Build args
|
||||||
|
|
||||||
|
| Arg | Default | Description |
|
||||||
|
|-----|---------|-------------|
|
||||||
|
| `NODE_MAJOR` | `24` | Node.js major version (matches Node base image) |
|
||||||
|
| `PNPM_VERSION` | `10.15.0` | pnpm version (corepack-activated) |
|
||||||
|
| `BUF_VERSION` | `1.55.0` | buf CLI version |
|
||||||
|
| `PROTOC_GEN_ES_VERSION` | `2.4.0` | `@bufbuild/protoc-gen-es` version (bump in lockstep with frontend's `@bufbuild/protobuf` runtime) |
|
||||||
|
| `ORAS_VERSION` | `1.2.2` | ORAS CLI version |
|
||||||
|
|
||||||
|
## Usage in CI
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
jobs:
|
||||||
|
test-unit:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
container:
|
||||||
|
image: 192.168.1.72/kollect-tools/ci/frontend-builder:latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v6
|
||||||
|
- run: pnpm install --frozen-lockfile
|
||||||
|
working-directory: frontend/svelte
|
||||||
|
- run: pnpm test
|
||||||
|
working-directory: frontend/svelte
|
||||||
|
```
|
||||||
|
|
||||||
|
For protobuf TS codegen (`buf.gen.yaml` declares `local: protoc-gen-es`):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- name: Generate proto TS bindings
|
||||||
|
working-directory: proto
|
||||||
|
run: buf generate
|
||||||
|
```
|
||||||
|
|
||||||
|
For Playwright E2E (drop `--with-deps` because system libs are pre-installed):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- name: Install Playwright browsers
|
||||||
|
working-directory: frontend/svelte
|
||||||
|
run: pnpm exec playwright install chromium
|
||||||
|
```
|
||||||
|
|
||||||
|
## Maintenance
|
||||||
|
|
||||||
|
Rebuild on dependency bumps:
|
||||||
|
|
||||||
|
- New Node major → bump `NODE_MAJOR`, push image
|
||||||
|
- pnpm bump → bump `PNPM_VERSION`, push image
|
||||||
|
- buf protocol changes → bump `BUF_VERSION` and `PROTOC_GEN_ES_VERSION`, push image
|
||||||
|
|
||||||
|
The CI workflow's weekly schedule rebuilds the image to keep base layers patched.
|
||||||
@@ -1,7 +1,8 @@
|
|||||||
# KollectAI CI — Java Builder Image
|
# KollectAI CI — Java Builder Image
|
||||||
#
|
#
|
||||||
# Pre-baked build environment for backend + plugin CI jobs.
|
# Pre-baked build environment for backend + plugin CI jobs.
|
||||||
# Contains: Java 21, Maven 3.9.9, common dependencies, OWASP NVD database.
|
# Contains: Java 21, Maven 3.9.9, Node.js + pnpm, buf CLI, OWASP NVD
|
||||||
|
# database, ORAS CLI, common dependencies.
|
||||||
#
|
#
|
||||||
# Build:
|
# Build:
|
||||||
# docker build -t 192.168.1.72/kollect-tools/ci/java-builder:latest ci/java-builder/
|
# docker build -t 192.168.1.72/kollect-tools/ci/java-builder:latest ci/java-builder/
|
||||||
@@ -79,7 +80,24 @@ ARG ORAS_VERSION=1.2.2
|
|||||||
RUN curl -fsSL "https://github.com/oras-project/oras/releases/download/v${ORAS_VERSION}/oras_${ORAS_VERSION}_linux_amd64.tar.gz" \
|
RUN curl -fsSL "https://github.com/oras-project/oras/releases/download/v${ORAS_VERSION}/oras_${ORAS_VERSION}_linux_amd64.tar.gz" \
|
||||||
| tar -xz -C /usr/local/bin oras
|
| tar -xz -C /usr/local/bin oras
|
||||||
|
|
||||||
|
# ─────────────────────────────────────────────────────────────────────
|
||||||
|
# buf CLI — single static binary used for `buf lint` in CI and pre-push
|
||||||
|
# parity. Backend Java codegen lives in the Maven build (protobuf-maven-
|
||||||
|
# plugin), not buf, so no protoc plugins are needed in this image.
|
||||||
|
# ─────────────────────────────────────────────────────────────────────
|
||||||
|
ARG BUF_VERSION=1.55.0
|
||||||
|
|
||||||
|
RUN curl -fsSL "https://github.com/bufbuild/buf/releases/download/v${BUF_VERSION}/buf-Linux-x86_64" \
|
||||||
|
-o /usr/local/bin/buf \
|
||||||
|
&& chmod +x /usr/local/bin/buf
|
||||||
|
|
||||||
WORKDIR /workspace
|
WORKDIR /workspace
|
||||||
|
|
||||||
# Verify installation
|
# Verify installation
|
||||||
RUN java -version && mvn -version && node --version && pnpm --version && oras version && jq --version
|
RUN java -version \
|
||||||
|
&& mvn -version \
|
||||||
|
&& node --version \
|
||||||
|
&& pnpm --version \
|
||||||
|
&& buf --version \
|
||||||
|
&& oras version \
|
||||||
|
&& jq --version
|
||||||
|
|||||||
@@ -9,9 +9,16 @@ Pre-baked build environment for KollectAI-ETL backend and plugin CI jobs.
|
|||||||
- Pre-cached Maven dependencies (Spring Boot, Flink, MyBatis, etc.)
|
- Pre-cached Maven dependencies (Spring Boot, Flink, MyBatis, etc.)
|
||||||
- Pre-installed `plugin-api` in local Maven repo
|
- Pre-installed `plugin-api` in local Maven repo
|
||||||
- OWASP NVD database snapshot
|
- OWASP NVD database snapshot
|
||||||
|
- Node.js + pnpm (via corepack)
|
||||||
|
- [buf](https://buf.build) CLI — for `buf lint` parity with pre-push
|
||||||
- ORAS CLI (Harbor artifact uploads)
|
- ORAS CLI (Harbor artifact uploads)
|
||||||
- git, jq, curl
|
- git, jq, curl
|
||||||
|
|
||||||
|
> **Note**: Backend Java protobuf codegen lives in the Maven build via
|
||||||
|
> `protobuf-maven-plugin`, not buf — so this image deliberately omits
|
||||||
|
> protoc plugins. For TypeScript proto codegen and frontend CI jobs use
|
||||||
|
> the [`frontend-builder`](../frontend-builder/) image.
|
||||||
|
|
||||||
## Build
|
## Build
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -23,10 +30,14 @@ docker push 192.168.1.72/kollect-tools/ci/java-builder:latest
|
|||||||
|
|
||||||
| Arg | Default | Description |
|
| Arg | Default | Description |
|
||||||
|-----|---------|-------------|
|
|-----|---------|-------------|
|
||||||
| `MAVEN_VERSION` | `3.9.9` | Maven version |
|
| `JAVA_VERSION` | `25` | Eclipse Temurin JDK version |
|
||||||
|
| `MAVEN_VERSION` | `3.9.14` | Maven version |
|
||||||
| `OWASP_DC_VERSION` | `12.1.1` | OWASP Dependency-Check version |
|
| `OWASP_DC_VERSION` | `12.1.1` | OWASP Dependency-Check version |
|
||||||
| `ETL_BRANCH` | `001-ai-etl-platform` | Branch to fetch pom.xml files from |
|
| `NODE_MAJOR` | `24` | Node.js major version |
|
||||||
|
| `PNPM_VERSION` | `10.15.0` | pnpm version (corepack-activated) |
|
||||||
|
| `BUF_VERSION` | `1.55.0` | buf CLI version |
|
||||||
| `ORAS_VERSION` | `1.2.2` | ORAS CLI version |
|
| `ORAS_VERSION` | `1.2.2` | ORAS CLI version |
|
||||||
|
| `NVD_API_KEY` | (empty) | Optional NVD API key — speeds up the OWASP database update during image build |
|
||||||
|
|
||||||
## Usage in CI
|
## Usage in CI
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user