You need the libXmu library – if it's already installed, you're probably missing the development headers (/usr/include/X11/Xmu and others), which many distributions require to be installed separately.
You will have to either compile libXmu the same way manually, or obtain the libxmu-dev or libxmu-devel package and extract it into, say, ~/usr.
Afterwards, set:
export CFLAGS="-I$HOME/usr/include" export LDFLAGS="-L$HOME/usr/lib"
You may also need:
export PKG_CONFIG_PATH="$HOME/usr/lib/pkgconfig:$HOME/usr/share/pkgconfig"
Also take a look at xsel, which does pretty much the same thing but requires less library dependencies.
Note: --prefix ~/bin/xclip will result in ugly paths such as ~/bin/xclip/bin/xclip and ~/bin/xclip/share/man/man1/xclip.1. Something like --prefix ~/usr will give nicer results.
This is especially worth noting when installing libraries; it is much easier to compile a program when all its libraries are in --prefix ~/usr, rather than each having its own --prefix.