Machine Learning

1742 readers

1 users here now

founded 4 years ago

MODERATORS

osipov@lemmy.ml

How to convert a positionally encoded predicted embedding from a decoder to its matching token? (infosec.pub)

submitted 1 month ago by yboutros@infosec.pub to c/machinelearning@lemmy.ml

0 comments fedilink hide all child comments

When training a transformer on positionally encoded embeddings, should the tgt output embeddings also be positionally encoded? If so, wouldn't the predicted/decoded embeddings also be positionally encoded?

no comments (yet)

sorted by: hot top controversial new old

there doesn't seem to be anything here