Opening a modded Minecraft server (Fabric, Forge)

Here's how to open a Java Edition server with mods. Installing Java, the firewall and connecting work the same as in Creating a Minecraft server (Java Edition, Paper and vanilla), so first complete step 1 (Install Java) of that guide.

Know this first

To join a modded server, the server and every player must install the same mod loader (Fabric or Forge), the same game version and the same mods. (Except for some server-only mods)
Mods use a lot of memory. For a modpack, we recommend a plan with 8GB or more. How to choose VPS specs (recommended specs by use case)
Plugins (for Paper) and mods are different things. If you'll only use plugins, see Tips for running a Minecraft server.

Fabric server

On the server install page at fabricmc.net, choose the game version and copy the download link for the executable server jar (Executable Server).

mkdir ~/fabric && cd ~/fabric
wget -O fabric-server.jar "copied download link"
java -Xms1G -Xmx6G -jar fabric-server.jar nogui

On the first run, it downloads the files it needs and then exits. Agree to the EULA.

sed -i 's/eula=false/eula=true/' eula.txt

Put mod files (.jar) in the mods folder. Most Fabric mods also need the Fabric API mod.

cd ~/fabric/mods
wget "mod download link"

Run it again.

Forge server

On files.minecraftforge.net, choose the game version and download the Installer. (To skip the ad page, use the direct link that appears when you click the small i icon next to Installer.)

mkdir ~/forge && cd ~/forge
wget -O forge-installer.jar "copied Installer link"
java -jar forge-installer.jar --installServer

When the installation finishes, run.sh is created. Memory settings go in user_jvm_args.txt.

echo "-Xms1G -Xmx6G" >> user_jvm_args.txt
./run.sh nogui

Agree to the EULA (sed -i 's/eula=false/eula=true/' eula.txt), put your mods in the mods folder, and run it again.

NeoForge is installed the same way (--installServer → run.sh), using the Installer from neoforged.net.

Where to get mods

Get them from Modrinth or CurseForge. When downloading, always check that the game version and the loader (Fabric/Forge) match your server. Don't install mods from unknown sources.

Keeping it running

Run it inside screen and it keeps going even after you close SSH. (How to use screen) For a script that starts it again automatically when it stops, and for automatic backups, you can use the methods in Tips for running a Minecraft server as they are. (Just replace the java ... server.jar line with your own start command.)

Common problems

The server shuts down as soon as it starts — Check the last error in the log (logs/latest.log). The most common causes are a mod for the wrong version or a missing mod it depends on (e.g., Fabric API).
An error saying the mods are different when joining — Make the mod list and versions on your computer exactly the same as the server's.
It's really laggy — Increase -Xmx or consider a plan with higher specs. Checking a slow server