Weaviate容器化部署：Docker与Kubernetes
在当今云原生时代，向量数据库的容器化部署已成为AI应用架构的核心需求。Weaviate作为开源向量数据库，其容器化部署方案为企业级AI应用提供了弹性扩展、高可用性和简化运维的完美解决方案。本文将深入探讨Weaviate在Docker和Kubernetes环境下的部署策略、最佳实践和运维技巧。## Weaviate架构概览```mermaidgraph TBsubgraph "W...
·
Weaviate容器化部署：Docker与Kubernetes
项目地址: https://gitcode.com/GitHub_Trending/we/weaviate 概述
在当今云原生时代，向量数据库的容器化部署已成为AI应用架构的核心需求。Weaviate作为开源向量数据库，其容器化部署方案为企业级AI应用提供了弹性扩展、高可用性和简化运维的完美解决方案。本文将深入探讨Weaviate在Docker和Kubernetes环境下的部署策略、最佳实践和运维技巧。
Weaviate架构概览
Docker单机部署
基础Docker部署
Weaviate提供了官方Docker镜像，支持快速单机部署：
# docker-compose.yml
version: '3.4'
services:
weaviate:
image: semitechnologies/weaviate:latest
ports:
- "8080:8080"
environment:
- QUERY_DEFAULTS_LIMIT=25
- AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED=true
- PERSISTENCE_DATA_PATH=/var/lib/weaviate
- DEFAULT_VECTORIZER_MODULE=none
volumes:
- weaviate_data:/var/lib/weaviate
restart: unless-stopped
volumes:
weaviate_data:
带向量化模块的部署
集成文本向量化模块的完整配置：
version: '3.4'
services:
weaviate:
image: semitechnologies/weaviate:latest
ports:
- "8080:8080"
environment:
- AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED=true
- PERSISTENCE_DATA_PATH=/var/lib/weaviate
- DEFAULT_VECTORIZER_MODULE=text2vec-transformers
- ENABLE_MODULES=text2vec-transformers
- TRANSFORMERS_INFERENCE_API=http://t2v-transformers:8080
depends_on:
- t2v-transformers
volumes:
- weaviate_data:/var/lib/weaviate
restart: unless-stopped
t2v-transformers:
image: semitechnologies/transformers-inference:sentence-transformers-multi-qa-MiniLM-L6-cos-v1
environment:
- ENABLE_CUDA=0
volumes:
weaviate_data:
环境变量配置详解
| 环境变量 | 描述 | 默认值 | 必需 |
|---|---|---|---|
PERSISTENCE_DATA_PATH |
数据持久化路径 | /var/lib/weaviate |
是 |
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED |
匿名访问 | false |
否 |
DEFAULT_VECTORIZER_MODULE |
默认向量化模块 | none |
否 |
ENABLE_MODULES |
启用模块列表 | - | 否 |
CLUSTER_HOSTNAME |
集群主机名 | - | 集群部署 |
Kubernetes集群部署
基础Kubernetes部署
创建Weaviate的Kubernetes部署配置：
# weaviate-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: weaviate
labels:
app: weaviate
spec:
replicas: 3
selector:
matchLabels:
app: weaviate
template:
metadata:
labels:
app: weaviate
spec:
containers:
- name: weaviate
image: semitechnologies/weaviate:latest
ports:
- containerPort: 8080
env:
- name: PERSISTENCE_DATA_PATH
value: "/var/lib/weaviate"
- name: AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED
value: "true"
- name: CLUSTER_HOSTNAME
valueFrom:
fieldRef:
fieldPath: status.podIP
volumeMounts:
- name: weaviate-data
mountPath: /var/lib/weaviate
resources:
requests:
memory: "2Gi"
cpu: "1"
limits:
memory: "4Gi"
cpu: "2"
volumes:
- name: weaviate-data
persistentVolumeClaim:
claimName: weaviate-pvc
Service和Ingress配置
# weaviate-service.yaml
apiVersion: v1
kind: Service
metadata:
name: weaviate-service
spec:
selector:
app: weaviate
ports:
- port: 8080
targetPort: 8080
type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: weaviate-ingress
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: "50m"
spec:
rules:
- host: weaviate.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: weaviate-service
port:
number: 8080
持久化存储配置
# weaviate-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: weaviate-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Gi
storageClassName: standard
高可用集群部署
多节点集群配置
# weaviate-cluster.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: weaviate-cluster
spec:
serviceName: "weaviate"
replicas: 3
selector:
matchLabels:
app: weaviate
template:
metadata:
labels:
app: weaviate
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- weaviate
topologyKey: "kubernetes.io/hostname"
containers:
- name: weaviate
image: semitechnologies/weaviate:latest
ports:
- containerPort: 8080
env:
- name: CLUSTER_GOSSIP_BIND_PORT
value: "7100"
- name: CLUSTER_DATA_BIND_PORT
value: "7101"
- name: CLUSTER_HOSTNAME
value: "$(HOSTNAME).weaviate.weaviate.svc.cluster.local"
- name: ENABLE_MODULES
value: "text2vec-transformers"
- name: DEFAULT_VECTORIZER_MODULE
value: "text2vec-transformers"
监控与运维
健康检查配置
livenessProbe:
httpGet:
path: /v1/.well-known/ready
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
httpGet:
path: /v1/.well-known/ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 3
资源监控配置
resources:
requests:
memory: "4Gi"
cpu: "2"
limits:
memory: "8Gi"
cpu: "4"
安全配置
TLS证书配置
env:
- name: TLS_CERTIFICATE
value: "/etc/ssl/certs/tls.crt"
- name: TLS_PRIVATE_KEY
value: "/etc/ssl/private/tls.key"
- name: TLS_VERIFY_CLIENT
value: "require"
认证配置
env:
- name: AUTHENTICATION_OIDC_ENABLED
value: "true"
- name: AUTHENTICATION_OIDC_ISSUER
value: "https://auth.example.com"
- name: AUTHENTICATION_OIDC_CLIENT_ID
value: "weaviate-client"
部署策略对比
| 部署方式 | 适用场景 | 优点 | 缺点 |
|---|---|---|---|
| Docker单机 | 开发测试、小型项目 | 部署简单、资源消耗低 | 单点故障、扩展性差 |
| Kubernetes单实例 | 生产环境入门 | 高可用、易于管理 | 配置相对复杂 |
| Kubernetes集群 | 企业级生产 | 弹性扩展、自动恢复 | 运维复杂度高 |
故障排除指南
常见问题及解决方案
| 问题现象 | 可能原因 | 解决方案 |
|---|---|---|
| 容器启动失败 | 端口冲突 | 检查8080端口占用情况 |
| 数据持久化失败 | 存储权限问题 | 检查PVC配置和存储类 |
| 向量化服务不可用 | 模块配置错误 | 验证ENABLE_MODULES设置 |
| 集群节点无法通信 | 网络策略限制 | 检查Kubernetes网络策略 |
性能优化建议
内存优化配置
env:
- name: MAX_IMPORT_GOROUTINES_FACTOR
value: "2"
- name: MAX_CONCURRENT_IMPORTS
value: "10"
- name: DISABLE_GRAPHQL_INTROSPECTION
value: "true"
查询优化配置
env:
- name: QUERY_MAXIMUM_RESULTS
value: "10000"
- name: QUERY_DEFAULTS_LIMIT
value: "100"
- name: QUERY_NESTED_CROSS_REFERENCE_LIMIT
value: "100"
总结
Weaviate的容器化部署为AI应用提供了强大的向量数据库基础设施。通过Docker可以快速搭建开发测试环境，而Kubernetes则为企业级生产环境提供了完整的解决方案。合理的资源配置、监控策略和安全措施是确保Weaviate稳定运行的关键。随着云原生技术的不断发展，Weaviate在容器化环境中的部署将变得更加简单和高效。
选择合适的部署方案需要根据具体的业务需求、团队技术栈和运维能力进行综合考量。无论是简单的Docker部署还是复杂的Kubernetes集群，Weaviate都能为您的AI应用提供可靠的向量搜索服务。
火山引擎开发者社区是火山引擎打造的AI技术生态平台，聚焦Agent与大模型开发，提供豆包系列模型（图像/视频/视觉）、智能分析与会话工具，并配套评测集、动手实验室及行业案例库。社区通过技术沙龙、挑战赛等活动促进开发者成长，新用户可领50万Tokens权益，助力构建智能应用。
更多推荐
27
0- 0
已为社区贡献26条内容
所有评论(0)