Mmm, some toughts from the stream:
- I would keep the spellbook only to store spells known, not to use them.
- I would replace prepared spells entirely with the spelldeck.
- I would add a minimum and maximum number of cards in the spelldeck, that you can "build" (or "rebuild") at the end of a long rest.
- I would add a "hand of cards" that you draw at the beginning (of what? Of the combat? Of the day? Each time you need to cast a spell?) and from which you can choose.
- I would add some special non-spell cards to mix inside the spelldeck (to draw, discard, shaffle, apply metamagic-like effects...), that you unlock as you level up as features, and that you can play using the relative action (bonus or "standard", depending on the card).
- The down/upscaling mechanic is interesting, but it needs to be tested. If you introduce the "hand of cards" you could probably remove this mechanic.
More than anything, i can't see the theme of the contest in this subclass (but that's probably because I haven't figured it out yet)