The size limit is currently set to 4GB, but I don’t recommend uploading large amount of videos to Obsidian Publish because it’s not optimized for videos - Depending on your student’s locations and network conditions, the videos may not play very well.
Dedicated video hosting services like YouTube and Vimeo optimize their infrastructure to host and deliver videos, they can re-encode your videos in more compact formats, switch delivery rates on the fly depending on network conditions, etc. Highly recommend using one of those and use <iframe> to embed them in your site.