K3s docker.io 国内镜像配置无效

K3s
1

环境信息:
K3s 版本:

v1.29.7+k3s1
节点 CPU 架构、操作系统和版本：:

Linux 158 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
集群配置:

1 servers,1agents
问题描述:

配置 /etc/rancher/k3s/registries.yml 后 crictl info中mirrors还是null
参考https://forums.rancher.cn/t/k3s-registries-yaml/2993 中查看 hosts中已经是对应配置
但是镜像还是从docker.com拉取

复现步骤:

  • 安装 K3s 的命令:
cat /etc/rancher/k3s/registries.yaml
mirrors:
  docker.io:
    endpoint:
      - "https://docker.moghn.eu.org/"
      - "https://registry.cn-hangzhou.aliyuncs.com/"

 crictl info |grep -A 5 regis
    "registry": {
      "configPath": "/var/lib/rancher/k3s/agent/etc/containerd/certs.d",
      "mirrors": null,
      "configs": null,
      "auths": null,
      "headers": null
--
    "sandboxImage": "registry.cn-hangzhou.aliyuncs.com/rancher/mirrored-pause:3.6",
    "statsCollectPeriod": 10,
    "systemdCgroup": false,
    "enableTLSStreaming": false,
    "x509KeyPairStreaming": {
      "tlsCertFile": "",


cat /var/lib/rancher/k3s/agent/etc/containerd/certs.d/docker.io/hosts.toml
# File generated by k3s. DO NOT EDIT.

server = "https://registry-1.docker.io/v2"
capabilities = ["pull", "resolve", "push"]


[host]
[host."https://docker.moghn.eu.org/v2"]
  capabilities = ["pull", "resolve"]
[host."https://registry.cn-hangzhou.aliyuncs.com/v2"]
  capabilities = ["pull", "resolve"]

预期结果:

实际结果:

附加上下文/日志:

日志 
Failed to pull image "apache/hertzbeat-collector:v1.6.0": failed to pull and unpack image "docker.io/apache/hertzbeat-collector:v1.6.0": failed to copy: httpReadSeeker: failed open: failed to do request: Get "https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/15/153dca8ae5a2b7466116bd04a34053b6485a43789cbea770f9514662fa50d41a/data?verify=1724296407-eCJGrPZQ%2BvMDW2aPlSiCXBNWKuo%3D": dial tcp 108.160.165.212:443: connect: connection timed out
2

你的配置是没问题的，至于 “crictl info中mirrors还是null” 的原因，你应该已经知道了。

按照你的配置，如果你从 docker.io 去拉取镜像，会默认跳到 第一个 endpoint，如果找不到，再往下继续查找，大概是这样的流程。

如果你的 mirror 可用，基本是可以找到镜像的。而且拉取的镜像，就是 docker.io 开头的，例如：docker.io/library/alpine

有一个常见的情况，会导致你配置的没生效。就是 你本地已经安装过 docker 或者 crictl，使用 crictl 拉取镜像的时候并不是使用 K3s 自带的那个，导致没加载 K3s 的一些配置。

你可以试试 k3s crictl pull busybox 这种，如果还是拉不下来，可以吧拉取失败的日志贴过来。

3

通过这个可以拉到
k3s crictl pull apache/hertzbeat
Image is up to date for sha256:bfbb35cd043491353dea568e497f398e16fd0ca3e92b5dfea67d9c5a1c268191

k3s crictl images
IMAGE TAG IMAGE ID SIZE
docker.io/apache/hertzbeat latest bfbb35cd04349 590MB
registry.cn-hangzhou.aliyuncs.com/rancher/klipper-helm v0.8.4-build20240523 0c1204a22f132 94.4MB
registry.cn-hangzhou.aliyuncs.com/rancher/klipper-lb v0.4.7 edc812b8e25d0 4.78MB
registry.cn-hangzhou.aliyuncs.com/rancher/local-path-provisioner v0.0.28 5d221316a3c61 18.4MB
registry.cn-hangzhou.aliyuncs.com/rancher/mirrored-coredns-coredns 1.10.1 ead0a4a53df89 16.2MB
registry.cn-hangzhou.aliyuncs.com/rancher/mirrored-library-traefik 2.10.7 ee69e8120b64a 43.2MB
registry.cn-hangzhou.aliyuncs.com/rancher/mirrored-metrics-server v0.7.0 b9a5a1927366a 19.4MB
registry.cn-hangzhou.aliyuncs.com/rancher/mirrored-pause 3.6 6270bb605e12e 301kB

4

那就是这个问题引起的

5

确实装了docker 我按照这个思路再看看 感谢

6

奇怪 还是不行
docker删除了
查看crictl 是只有一个
locate crictl
/usr/local/bin/crictl
/var/lib/rancher/k3s/agent/etc/crictl.yaml

7

那可以手动指定下 cri 的 config，例如：

export CRI_CONFIG_FILE=/var/lib/rancher/k3s/agent/etc/crictl.yaml
crictl ps
8

也可以 用helm 或者kuboard上操作就不行了

9

如果不行，你就把不行的上下文描述清楚，还有日志，一句话很难猜出来原因

10

我在看看

11

在镜像源配置好得情况下通过crictl pull 镜像是可以拉取到得
安装 helm 跟 kuboard 从helm 部署应用
然后从kuboard 查看容器状态 都是报错

Failed to pull image "apache/hertzbeat-collector:v1.6.0": failed to pull and unpack image "docker.io/apache/hertzbeat-collector:v1.6.0": failed to copy: httpReadSeeker: failed open: failed to do request: Get "https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/15/153dca8ae5a2b7466116bd04a34053b6485a43789cbea770f9514662fa50d41a/data?verify=1724296407-eCJGrPZQ%2BvMDW2aPlSiCXBNWKuo%3D": dial tcp 108.160.165.212:443: connect: connection timed out

从kuboard上手动手动添加容器得话 ，也是这样报错

12

理论上不太可能，虽然我没用过 kuboard ，:smile:

后者你从 kubectl 部署个 yaml 试下

13

kuboard用 yaml 跑的 也是可以 :rofl:

14

发现 agent 不走镜像地址 server是按镜像地址走的

15

k3s-agent 也需要配置 registries.yaml

16

是的，agent 也需要配置

17

版本：v1.30.4+k3s1 (98262b5d)；和楼主一样配置了国内镜像，我也是没法拉取镜像，使用k3s crictl pull busybox 会出现如下错误信息（本地没有docker）：

E0904 11:24:50.951856 1711 remote_image.go:180] “PullImage from image service failed” err=“rpc error: code = FailedPrecondition desc = failed to pull and unpack image "docker.io/library/busybox:latest": failed commit on ref "unknown-sha256:b6f90960b46d883eb6835a39284d6a43f64b09cd62303aab36cd625e52cf03e9": "unknown-sha256:b6f90960b46d883eb6835a39284d6a43f64b09cd62303aab36cd625e52cf03e9" failed size validation: 8177 != 7828: failed precondition” image=“busybox”
FATA[0000] pulling image: rpc error: code = FailedPrecondition desc = failed to pull and unpack image “docker.io/library/busybox:latest”: failed commit on ref “unknown-sha256:b6f90960b46d883eb6835a39284d6a43f64b09cd62303aab36cd625e52cf03e9”: “unknown-sha256:b6f90960b46d883eb6835a39284d6a43f64b09cd62303aab36cd625e52cf03e9” failed size validation: 8177 != 7828: failed precondition

18

你是咋配置的？

19 
# cat /etc/rancher/k3s/registries.yaml 
mirrors:
  "docker.io":
    endpoint:
      - "http://hub-mirror.c.163.com" # 可根据需求替换 mirror 站点
      - "https://docker.mirrors.ustc.edu.cn"
      - "https://cr.console.aliyun.com/"

# cat /var/lib/rancher/k3s/agent/etc/containerd/certs.d/docker.io/hosts.toml
# File generated by k3s. DO NOT EDIT.

server = "https://registry-1.docker.io/v2"
capabilities = ["pull", "resolve", "push"]


[host]
[host."http://hub-mirror.c.163.com/v2"]
  capabilities = ["pull", "resolve"]
[host."https://docker.mirrors.ustc.edu.cn/v2"]
  capabilities = ["pull", "resolve"]
[host."https://cr.console.aliyun.com/v2"]
  capabilities = ["pull", "resolve"]
20

配置没问题，如果所有节点都配置了，理论上不应该