Are you planning to run or fine-tuning , and do you have a specific GPU cluster configuration in mind? Installation — Megatron Core - NVIDIA Documentation
Note: If the installation fails due to memory issues during compilation, limit the parallel jobs by prefixing the command with MAX_JOBS=4 . Download Install Megatron Repo zip
Once inside the container, you can proceed with the source installation as described above. Are you planning to run or fine-tuning ,
The NVIDIA Megatron Core Documentation warns that this step may take 20+ minutes as it compiles CUDA kernels. Download Install Megatron Repo zip