Slavery wasn't a liberal or conservative viewpoint either.
The act or support of extending rights/liberty to those without those rights is a liberal, progressive action. Abolition was a federalist cause meaning they were looking to the federal government to take action to end slavery in all states.
Then, simply look at the definitions of the words...
CONSERVATIVE - con·serv·a·tive
–adjective
From the Latin
servare - meaning "keep, preserve."
1. disposed to preserve existing conditions, institutions, etc., or to restore traditional ones, and to limit change.
2. cautiously moderate or purposefully low: a conservative estimate.
3. traditional in style or manner; avoiding novelty or showiness: conservative suit.
–noun: a person who is conservative in principles, actions, habits, etc.
Slavery was the existing condition...
LIBERAL - lib·er·al
–adjective
From the Latin
liberalis - meaning "suitable for a freeman."
1. favorable to progress or reform, as in political or religious affairs.
2. (often initial capital letter) noting or pertaining to a political party advocating measures of progressive political reform.
3. of, pertaining to, based on, or advocating liberalism.
4. favorable to or in accord with concepts of maximum individual freedom possible, esp. as guaranteed by law and secured by governmental protection of civil liberties.
5. favoring or permitting freedom of action, esp. with respect to matters of personal belief or expression: a liberal policy toward dissident artists and writers.
6. of or pertaining to representational forms of government rather than aristocracies and monarchies.
7. free from prejudice or bigotry; tolerant: a liberal attitude toward foreigners.
8. open-minded or tolerant, esp. free of or not bound by traditional or conventional ideas, values, etc.
9. characterized by generosity and willingness to give in large amounts: a liberal donor.
10. given freely or abundantly; generous: a liberal donation.
11. not strict or rigorous; free; not literal: a liberal interpretation of a rule.
12. of, pertaining to, or based on the liberal arts.
13. of, pertaining to, or befitting a freeman.
–noun:. a person of liberal principles or views, esp. in politics or religion.