As nouns,
system of numeration is a hyponym of
mathematical notation; that is, system of numeration is a
word with a more specific, narrower meaning than
mathematical notation:
mathematical notation: a notation used by mathematicians
system of numeration: any notation for the representation of numbers