Pre-heat the oven to 180°C (convention)/160°C (fan-forced/convection).
Place butter and sugar in the bowl of an electric mixer. Beat for a few minutes until pale and fluffy. Gradually add the eggs, vanilla, cardamom and ginger and continue beating for 2-3 more minutes.
Add rosemary, rye flour, almond meal, almond flakes, baking powder and salt to the egg mixture in the bowl. Using a wooden or silicone spoon, mix the batter gently by hand until just but thoroughly combined. Make sure there are no dry flour bits or wet eggy mixture at the bottom of the bowl (Do not overmix).
Grease and line 2 mini loaf pans (15.5cm X 9cm) or 1 standard loaf pan (21cm X 11cm) with baking paper. If using 2 mini loaf pans, divide batter equally between the loaf pans filling each pan 2/3rds to the top. If using 1 standard loaf pan, spoon all the batter in the one loaf pan.
Using a sharp knife, cut each quarter of the poached quince slice into three narrow slices. Top the cake batter with the quince slices, placing the slices at equal intervals. Sprinkle the extra sugar and rosemary leaves on top.
Bake in the preheated oven for 50-60 minutes until starting to brown around the edges and cooked through the centre when tested with a skewer down the middle.
Remove from oven and cool completely before slicing with a sharp knife so as to cut cleanly through the quince. Enjoy as-is or warmed up with a bit of clotted cream.
Notes
If you don't have quince and would still like to make this loaf, lightly poach a whole peeled apple or pear in a heavy-bottomed saucepan submerged in 2 cups water, 1/2 cup fruity red wine, orange peels, cinnamon stick, star anise, dash of honey and 1/2 cup sugar for 30 minutes on medium heat. Then cool and cut into slices. Use those slices in this recipe instead of quince. But honestly, look for quinces, as they are divine!