All dates in Gramps are internally converted to Modified Julian Date so that they can be compared no matter the calendar (Gregorian, Julian, Hebrew, Islamic, Persian, French Republican, Swedish). This is done in file gramps/gen/lib/gcalendar.py. Look at the xxx_sdn functions (sdn = Serial Date Number).
Be cautious, though. There is an undocumented trick to make computations easier. As an intermediate step, all years are offset to start on March 1st so that any year is made of 2 identical 5-month sequences followed by 2 “anomalous” months. Of course an inverse offset is applied at end.
You can add your own calendar by writing two conversions methods (to/from Julian).
You also need to write a date parser (gramps/gen/datehandler/_dateparser.py) so that what you type can be passed to the date conversion methods from what is entered with the GUI. You perhaps also need a date formatter (gramps/gen/datehandler/_datedisplay.py) to output an internal date in human-readable form, though the Date object keeps the original year, month, day elements in some fields. This supposes your fictitious calendar has the same structure: 3 components equivalent to year, month and day.
The other _date_xx.py files in gramps/gen/datehandler/ are for localisation. If you’re using the US version of Gramps, you won’t need them.
As already mentioned by others, Gramps is written in Python. The link provided by @anon9092048 is about date localisation in languages other than en_US.
My recommendation would be to study first Gramps code. For that, download Gramps source and install LXR (click on the SourceForge logo to access the download service). It will allow you to browse the code with your usual browser. Identifiers are clickable and give you access to cross-references and you can jump quickly to these references.
Documentation for Python is available here. Note these aren’t tutorials so you need intellectual effort to link with your own questions about the code.
Try to integrate with the present data management. Don’t attempt to design your own GUI dialogs. This is possible is you stick with the Y-M-D paradigm which is presently the basis of all dates in Gramps. If you really want to create your own dialogs, you’ll be faced with a tremendous task needing to understand control flow inside Gramps. This is daunting task. Gramps uses GTK+ widgets and relies heavily on “signals and slots”. You’ll find GTK3 general documentation here. This is absolute no tutorial and is of any help if you already have any notion about GTK+.