![]() ![]() Python train.py config/train_shakespeare_char.py -dataset=shakespeare 420.38s user 105.07s system 49% cpu 17:34.84 total ![]() Overriding: gradient_accumulation_steps = 1 Has someone tried 'mps' together with 'compile=True' and succeed? You'll have done as good to them be to brief ![]() Your people have are endured with them not: Your brats bear betwixt them away, and nothingĪgainst the gracious patern of their heads,įor their father is not their silly mouths, Since the common people'd courtesy 'gainst their times, The month of his son bounded bones and rough The like order precious soner stout the morning's strength Loading meta from data/shakespeare_char/meta.pkl. Flash Attention atm needs PyTorch nightly and dropout=0.0 Overriding: out_dir = out-shakespeare-char Python sample.py -out_dir=out-shakespeare-char Warmup_iters = 100 # not super necessary potentiallyĬompile = False # do not torch compile the modelįound vocab_size = 65 (inside data/shakespeare_char/meta.pkl) Min_lr = 1e-6 # learning_rate / 10 usuallyīeta2 = 0.999 # make a bit bigger because number of tokens per iter is small Lr_decay_iters = 5000 # make equal to max_iters usually Learning_rate = 1e-3 # with baby networks can afford to go a bit higher ![]() Wandb_log = False # override via command line if you likeīlock_size = 256 # context of up to 256 previous characters # we expect to overfit on this small dataset, so only save when val improves Log_interval = 10 # don't print too too often # good for debugging and playing on macbooks and suchĮval_interval = 250 # keep frequent because we'll overfit # train a miniature character-level shakespeare model Overriding config with config/train_shakespeare_char.py: Python train.py config/train_shakespeare_char.py Does anyone have ideas what can be wrong here? Spent couple hours trying to reinstall everything but it didn't help. Defaulting to vocab_size of GPT-2 to 50304 (50257 rounded up for efficiency) ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |