Skip to content

FSQ implementations' bug #94

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
scenarios opened this issue Dec 15, 2023 · 4 comments
Closed

FSQ implementations' bug #94

scenarios opened this issue Dec 15, 2023 · 4 comments

Comments

@scenarios
Copy link

image

Your implementation on bound function might be incorrect.
tan() is not the inverse of tanh().

@lucidrains
Copy link
Owner

ah yes, i see it

the reason is bc it was transcribed from the pseudocode in the paper

Screen Shot 2023-12-15 at 6 36 48 AM

@fab-jul do you have any thoughts?

@lucidrains
Copy link
Owner

#95 will merge if author confirms this is an error

@fab-jul
Copy link

fab-jul commented Dec 16, 2023

Hi yes this is a mistake, should be arctanh.

Note that it barely matters as it's called with a value around 0, where both are similar.

Also note that the whole bound function is really ad-hoc and should be revisited. for example, maybe it should be symmetric around 0 also for even numbers. If anyone has some insights let me know.

@lucidrains
Copy link
Owner

@fab-jul yea i figured as i hear from a lot of people having success with FSQ! let me get this merged in just for correctness sake

@scenarios thank you for spotting this

@sekstini sekstini mentioned this issue Jan 18, 2024
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants