1. torch.library’ has no attribute 'custom_op

 Failed to import diffusers.pipelines.pipeline_utils because of the following error (look up to see its traceback):
Failed to import diffusers.models.autoencoders.autoencoder_kl because of the following error (look up to see its traceback):
module 'torch.library' has no attribute 'custom_op'

需要torch>=2.5

2. RuntimeError: r.nvmlDeviceGetNvLinkRemoteDeviceType_ INTERNAL ASSERT FAILED at

• CUDA driver version >= 470.42.01 or downgrade pytorch version <= 2.2.2
• 也可能只是单纯的显存超了

3. version `GLIBC_2.25‘ not found

  File "/miniconda3/envs/flux_cu124_pt260/lib/python3.10/site-packages/zmq/backend/cython/__init__.py", line 6, in <module>
    from . import _zmq
ImportError: /usr/lib64/libc.so.6: version `GLIBC_2.25' not found (required by /miniconda3/envs/flux_cu124_pt260/lib/python3.10/site-packages/zmq/backend/cython/../../../pyzmq.libs/libsodium-19479d6d.so.26.2.0)

这个报错的原因是zmq编译时链接的glibc的version高于当前系统中提供的glibc，但是zmq时pip安装时编译的，因此我们可以卸载重新安装即可

1）解决策略一

pip uninstall pyzmq
pip install pyzmq      # 会重新编译

这个问题一般发生在同一个环境在多机进行应用迁移时发生，注意这个包不止以来GLIBC还有GLIBCXX，很多时候是这两个一高一低，搞出好多问题。

我们详细分析这个问题

# 查看编译的zmq文件链接的文件路径
pip show pyzmq | grep Location   # site-packages
ldd site-packages/zmq/backend/cython/_zmq.cpython-310-x86_64-linux-gnu.so

# 查看系统glibc支持的版本
strings /usr/lib64/libstdc++.so.6 | grep GLIBC
strings /usr/lib64/libc.so.6 | grep GLIBC  # 应当时低于GLIBC_2.25的

        linux-vdso.so.1 =>  (0x00007fff7eeca000)
        /$LIB/libtdsp_block.so => /lib64/libtdsp_block.so (0x00007fba9b784000)
        libstdc++.so.6 => /usr/local/gcc/lib64/libstdc++.so.6 (0x00007fba9b36c000)
        libpthread.so.0 => /usr/lib64/libpthread.so.0 (0x00007fba9b150000)
        librt.so.1 => /usr/lib64/librt.so.1 (0x00007fba9af48000)
        libm.so.6 => /usr/lib64/libm.so.6 (0x00007fba9ac46000)
        libgcc_s.so.1 => /usr/local/gcc/lib64/libgcc_s.so.1 (0x00007fba9aa28000)
        libc.so.6 => /usr/lib64/libc.so.6 (0x00007fba9a65a000)
        libdl.so.2 => /usr/lib64/libdl.so.2 (0x00007fba9a

2）解决策略二：重装glibc

# 不要在ceph盘编译，否则很慢
# https://ftp.gnu.org/gnu/glibc/
wget https://ftp.gnu.org/gnu/glibc/glibc-2.35.tar.gz
tar -zxvf glibc-2.35.tar.gz
cd glibc-2.35
mkdir build
cd build
../configure  --prefix=/usr/local/glibc-2.35 --disable-profile --enable-add-ons --with-headers=/usr/include --with-binutils=/usr/bin
# ../configure  --prefix=/usr --disable-profile --enable-add-ons --with-headers=/usr/include --with-binutils=/usr/bin
# export LD_LIBRARY_PATH=.../usr/local/gcc/lib:/usr/local/gcc/lib64: =>
# export LD_LIBRARY_PATH=.../usr/local/gcc/lib:/usr/local/gcc/lib64
make # 不能 make -j8, job server eror, take ~
make install

安装make（机器不能获取最新版的make）

https://ftp.gnu.org/gnu/make/make-4.4.tar.gz
tar -zxvf make-4.4.tar.gz 
cd make-4.4
./configure --prefix=/usr
type make
# make check
make install
make -v