글

docker 실행 후 random 시간이 흐르고 나면 "Failed to initialize NVML: Unknown Error"가 나는 현상

 Possible fix list https://github.com/NVIDIA/nvidia-docker/issues/1671 https://gist.github.com/gengwg/55b3eb2bc22bcbd484fccbc0978484fc https://bbs.archlinux.org/viewtopic.php?id=266915 https://github.com/NVIDIA/nvidia-docker/issues/1447 1) 문제 재구현: 아래 solution으로 해결 가능한 문제인지 진단     - docker container 재시작 후 nvidia-smi 정상 동작 확인     - 호스트 측 터미널에 "systemctl daemon-reload" 입력후 container 안에서 nvidia-smi error 발생 확인 2) 해결 방법: docker run 인자 수정     - /dev/ 하위에 nvidia가 붙은 모든 instance들을 인자에 포함시켜 전달     - 예시: docker run -it --name detr --gpus all -v /home/work/Desktop/:/data --shm-size 128G -p 11022:22 -p 11006:6006 --device /dev/nvidia0:/dev/nvidia0 --device /dev/nvidia1:/dev/nvidia1 --device /dev/nvidia2:/dev/nvidia2 --device /dev/nvidia3:/dev/nvidia3 --device /dev/nvidia4:/dev/nvidia4 --device /dev/nvidia5:/dev/nvidia5 --device /dev/nvidia6:/dev/nvidia6 --device /dev/nvidia7:/dev/nvidia7 --device /dev/nvidia-caps/ --device /dev/nvidiactl --device /dev/nv...

tar 분할 압축 및 해제

 tar cvf some_files.tar some_files split -b 1000M some_files.tar "some_files.tar.part" cat some_files.tar.parta* > some_files.tar ref: https://www.tecmint.com/split-large-tar-into-multiple-files-of-certain-size/

ubuntu gui backend 설정

서버 이용시 gui 불가 ->  export MPLBACKEND=Agg

Disable Nouveau and install nvidia driver via runfile

 ref:  https://linuxconfig.org/how-to-disable-blacklist-nouveau-nvidia-driver-on-ubuntu-20-04-focal-fossa-linux https://linuxconfig.org/how-to-install-the-nvidia-drivers-on-ubuntu-20-04-focal-fossa-linux sudo bash -c "echo blacklist nouveau > /etc/modprobe.d/blacklist-nvidia-nouveau.conf" sudo bash -c "echo options nouveau modeset=0 >> /etc/modprobe.d/blacklist-nvidia-nouveau.conf" cat /etc/modprobe.d/blacklist-nvidia-nouveau.conf sudo update-initramfs -u sudo reboot sudo apt install build-essential libglvnd-dev pkg-config sudo telinit 3 CTRL + ALT + F1 sudo telinit 5 sudo apt install gcc-12 sudo ln -s -f /usr/bin/gcc-12 /usr/bin/gcc sudo bash NVIDIA-Linux-x86_64-440.44.run sudo reboot

docker cli 한글

apt-get install locales export LANGUAGE=ko_KR.UTF- 8 export LANG=ko_KR.UTF- 8 source ~/.bashrc locale-gen ko_KR ko_KR.UTF-8 update-locale LANG=ko_KR.UTF-8 dpkg-reconfigure locales ref:  https://proni.tistory.com/entry/%F0%9F%90%B3-Docker-%ED%95%9C%EA%B8%80-%EC%84%A4%EC%A0%95-%ED%95%9C%EA%B8%80-%EA%B9%A8%EC%A7%90-%ED%95%B4%EA%B2%B0%ED%95%98%EA%B8%B0

cp with progress / verify

  rsync -ah --progress source-file destination-file **주의: dest를 폴더명으로 지정하지 않도록 하기 ref:  https://askubuntu.com/questions/17275/how-to-show-the-transfer-progress-and-speed-when-copying-files-with-cp verfication: diff -rq --no-dereference /path/to/old/drive/ /path/to/new/drive/ ref: https://unix.stackexchange.com/questions/313089/verifying-a-large-directory-after-copy-from-one-hard-drive-to-another To remote server: rsync -avz -e ssh SOURCE USERNAME@IPADDRESS:TARGET

최근 docker container에서 apt update를 할때 나는 nvidia gpg key error 해결

rm /etc/apt/sources.list.d/cuda.list rm /etc/apt/sources.list.d/nvidia-ml.list apt-key del 7fa2af80 apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/7fa2af80.pub ref: https://github.com/NVIDIA/nvidia-docker/issues/1631