Skip to content

Navigation Menu

Appearance settings

View all features
- BY COMPANY SIZE
  Enterprises
  Small and medium teams
  Startups
  Nonprofits
- BY USE CASE
  App Modernization
  DevSecOps
  DevOps
  CI/CD
  View all use cases
- BY INDUSTRY
  Healthcare
  Financial services
  Manufacturing
  Government
  View all industries
View all solutions
- EXPLORE BY TOPIC
  AI
  Software Development
  DevOps
  Security
  View all topics
- EXPLORE BY TYPE
  Customer stories
  Events & webinars
  Ebooks & reports
  Business insights
  GitHub Skills
- SUPPORT & SERVICES
  Documentation
  Customer support
  Community forum
  Trust center
  Partners
View all resources
- COMMUNITY
  GitHub SponsorsFund open source developers
- PROGRAMS
  Security Lab
  Maintainer Community
  Accelerator
  GitHub Stars
  Archive Program
- REPOSITORIES
  Topics
  Trending
  Collections
- ENTERPRISE SOLUTIONS
  Enterprise platformAI-powered developer platform
- AVAILABLE ADD-ONS
  GitHub Advanced SecurityEnterprise-grade security features
  Copilot for BusinessEnterprise-grade AI features
  Premium SupportEnterprise-grade 24/7 support
Pricing

Search code, repositories, users, issues, pull requests...

Search

Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our documentation.

Appearance settings

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

huggingface / transformers Public

Notifications You must be signed in to change notification settings
Fork 33k
Star 160k

Code
Issues 1.1k
Pull requests 1.3k
Actions
Projects
Security and quality
Insights

Additional navigation options

Code
Issues
Pull requests
Actions
Projects
Security and quality
Insights

Reformer#3351

Merged

patrickvonplaten merged 162 commits intomasterhuggingface/transformers:masterfrom

reformer_add_modelhuggingface/transformers:reformer_add_modelCopy head branch name to clipboard

May 7, 2020

Conversation Commits162 (162)Checks Files changed

Merged

Reformer#3351
patrickvonplaten merged 162 commits intomasterfrom
reformer_add_model

Commits

Commits on May 7, 2020

first copy & past commit from Bert and morgans LSH code
patrickvonplaten
committed
add easy way to compare to trax original code
patrickvonplaten
committed
translate most of function
patrickvonplaten
committed
make trax lsh self attention deterministic with numpy seed + copy paste code
patrickvonplaten
committed
add same config
patrickvonplaten
committed
add same config
patrickvonplaten
committed
fix merge conflicts
patrickvonplaten
committed
make layer init work
patrickvonplaten
committed
implemented hash_vectors function for lsh attention
patrickvonplaten
committed
continue reformer translation
patrickvonplaten
committed
hf LSHSelfAttentionLayer gives same output as trax layer
patrickvonplaten
committed
refactor code
patrickvonplaten
committed
refactor code
patrickvonplaten
committed
refactor code
patrickvonplaten
committed
refactor
patrickvonplaten
committed
refactor + add reformer config
patrickvonplaten
committed
delete bogus file
patrickvonplaten
committed
split reformer attention layer into two layers
patrickvonplaten
committed
fix merge conflicts
patrickvonplaten
committed
save intermediate step
patrickvonplaten
committed
save intermediate step
patrickvonplaten
committed
make test work
patrickvonplaten
committed
add complete reformer block layer
patrickvonplaten
committed
finish reformer layer
patrickvonplaten
committed
implement causal and self mask
patrickvonplaten
committed
clean reformer test and refactor code
patrickvonplaten
committed
update init
patrickvonplaten
committed
fix device for GPU
patrickvonplaten
committed
fix chunk length init for tests
patrickvonplaten
committed
include morgans optimization
patrickvonplaten
committed
improve memory a bit
patrickvonplaten
committed
improve comment
patrickvonplaten
committed
factorize num_buckets
patrickvonplaten
committed
better testing parameters
patrickvonplaten
committed
make whole model work
patrickvonplaten
committed
make lm model work
patrickvonplaten
committed
add t5 copy paste tokenizer
patrickvonplaten
committed
add chunking feed forward
patrickvonplaten
committed
clean config
patrickvonplaten
committed
add improved assert statements
patrickvonplaten
committed
make tokenizer work
patrickvonplaten
committed
improve test
patrickvonplaten
committed
correct typo
patrickvonplaten
committed
extend config
patrickvonplaten
committed
add complexer test
patrickvonplaten
committed
add new axial position embeddings
patrickvonplaten
committed
add local block attention layer
patrickvonplaten
committed
clean tests
patrickvonplaten
committed
refactor
patrickvonplaten
committed
better testing
patrickvonplaten
committed
save intermediate progress
patrickvonplaten
committed
clean test file
patrickvonplaten
committed
make shorter input length work for model
patrickvonplaten
committed
allow variable input length
patrickvonplaten
committed
refactor
patrickvonplaten
committed
make forward pass for pretrained model work
patrickvonplaten
committed
add generation possibility
patrickvonplaten
committed
finish dropout and init
patrickvonplaten
committed
make style
patrickvonplaten
committed
refactor
patrickvonplaten
committed
add first version of RevNet Layers
patrickvonplaten
committed
make forward pass work and add convert file
patrickvonplaten
committed
make uploaded model forward pass work
patrickvonplaten
committed
make uploaded model forward pass work
patrickvonplaten
committed
refactor code
patrickvonplaten
committed
add namedtuples and cache buckets
patrickvonplaten
committed
correct head masks
patrickvonplaten
committed
refactor
patrickvonplaten
committed
made reformer more flexible
patrickvonplaten
committed
make style
patrickvonplaten
committed
remove set max length
patrickvonplaten
committed
add attention masks
patrickvonplaten
committed
fix up tests
patrickvonplaten
committed
fix conflict
patrickvonplaten
committed
fix lsh attention mask
patrickvonplaten
committed
make random seed optional for the moment
patrickvonplaten
committed
improve memory in reformer
patrickvonplaten
committed
add tests
patrickvonplaten
committed
make style
patrickvonplaten
committed
make sure masks work correctly
patrickvonplaten
committed
detach gradients
patrickvonplaten
committed
save intermediate
patrickvonplaten
committed
correct backprob through gather
patrickvonplaten
committed
make style
patrickvonplaten
committed
change back num hashes
patrickvonplaten
committed
rename to labels
patrickvonplaten
committed
fix rotation shape
patrickvonplaten
committed
fix detach
patrickvonplaten
committed
update
patrickvonplaten
committed
fix trainer
patrickvonplaten
committed
fix backward dropout
patrickvonplaten
committed
make reformer more flexible
patrickvonplaten
committed
fix
patrickvonplaten
committed
fix
patrickvonplaten
committed
add tests for fixed seed in reformer layer
patrickvonplaten
committed
fix trainer typo
patrickvonplaten
committed
fix typo in activations
patrickvonplaten
committed
add fp16 tests
patrickvonplaten
committed
add fp16 training
patrickvonplaten
committed
support fp16
patrickvonplaten
committed
correct gradient bug in reformer
patrickvonplaten
committed
add fast gelu
patrickvonplaten
committed
re-add dropout for embedding dropout
patrickvonplaten
committed
better naming
patrickvonplaten
committed
better naming
patrickvonplaten
committed
renaming
patrickvonplaten
committed
finalize test branch
patrickvonplaten
committed
finalize tests
patrickvonplaten
committed
add more tests
patrickvonplaten
committed
finish tests
patrickvonplaten
committed
fix
patrickvonplaten
committed
fix type trainer
patrickvonplaten
committed
fix fp16 tests
patrickvonplaten
committed
fix tests
patrickvonplaten
committed
fix tests
patrickvonplaten
committed
fix tests
patrickvonplaten
committed
fix issue with dropout
patrickvonplaten
committed
fix dropout seeds
patrickvonplaten
committed
correct random seed on gpu
patrickvonplaten
committed
finalize random seed for dropout
patrickvonplaten
committed
finalize random seed for dropout
patrickvonplaten
committed
remove duplicate line
patrickvonplaten
committed
correct half precision bug
patrickvonplaten
committed
make style
patrickvonplaten
committed
refactor
patrickvonplaten
committed
refactor
patrickvonplaten
committed
docstring
patrickvonplaten
committed
remove sinusoidal position encodings for reformer
patrickvonplaten
committed
move chunking to modeling_utils
patrickvonplaten
committed
make style
patrickvonplaten
committed
clean config
patrickvonplaten
committed
make style
patrickvonplaten
committed
fix tests
patrickvonplaten
committed
fix auto tests
patrickvonplaten
committed
pretrained models
patrickvonplaten
committed
fix docstring
patrickvonplaten
committed
update conversion file
patrickvonplaten
committed
Update pretrained_models.rst
patrickvonplaten
committed
fix rst
patrickvonplaten
committed
fix rst
patrickvonplaten
committed
update copyright
patrickvonplaten
committed
fix test path
patrickvonplaten
committed
fix test path
patrickvonplaten
committed
fix small issue in test
patrickvonplaten
committed
include reformer in generation tests
patrickvonplaten
committed
add docs for axial position encoding
patrickvonplaten
committed
finish docs
patrickvonplaten
committed
Update convert_reformer_trax_checkpoint_to_pytorch.py
patrickvonplaten
committed
remove isort
patrickvonplaten
committed
include sams comments
patrickvonplaten
committed
remove wrong comment in utils
patrickvonplaten
committed
correct typos
patrickvonplaten
committed
fix typo
patrickvonplaten
committed
Update reformer.rst
patrickvonplaten
committed
applied morgans optimization
patrickvonplaten
committed
make style
patrickvonplaten
committed
make gpu compatible
patrickvonplaten
committed
remove bogus file
patrickvonplaten
committed
big test refactor
patrickvonplaten
committed
add example for chunking
patrickvonplaten
committed
fix typo
patrickvonplaten
committed
add to README
patrickvonplaten
committed

Footer

© 2026 GitHub, Inc.

Footer navigation

Terms
Privacy
Security
Status
Community
Docs
Contact

You can’t perform that action at this time.