Length of max position embeddings

by deleted - opened Dec 20, 2022

deleted

Hi,

I would like to know why, in the config, "max_position_embeddings" is set 514 and not 512. Is it a typo?

Thank you!

i think it's an historical artefact coming from RoBERTa

ie. it's not a typo

wissamantoun changed discussion status to closed Jun 23, 2025

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment