Remove a Ubuntu package

# sudo apt-get remove package

Including configuration stuff
# sudo apt-get remove --purge package