You can install this from your DVD.
The DVD is probably your best bet and this is the method I used in 10.3 Final.
Search for xfce in your software manager and install it.
If you can install it this way, ignore the rest of this page.
This is what I used in Alpha 5 and Beta 1:
xfce4-4.4.1-installer.run (local) / xfce4-4.4.1-installer.run (remote)
After downloading, open a terminal and type a few commands:
I was unable to install the first time because I tried adding in the enhancements and checked a few things... but if you run it with the default settings it should install without any problems.
I didn't attempt to build from source this time, but here it is if you want it.
xfce-4.4.1-src.tar.bz2 (local) / xfce-4.4.1-src.tar.bz2 (remote)