Expand contributing installation guide with fork and setup steps
Quote editable install and link contributing guide from install.md
Fix relative path of contributing link in install.md
The current state of the PEFT docs is not one of structure and I was constantly annoyed that whenever I wanted to change something there were several places that needed touching and they all felt disconnected. So this is my attempt at structuring the docs. Some of these ideas are quite old (discussed in 01/2025) but are still valid.
I've removed most of the code guides without replacement. That's not ideal, I think we should have code examples but I'm think they should be method-focused. Maybe one general example of a training workflow is sufficient because most methods follow the same scheme.
All details from the method guides (prompting, lora, oft/boft, etc.) are now integrated into the respective method pages instead. I would have hesitated to do this if these guides would have integrated information about the adapters but they didn't. I think it makes a lot more sense to have one place for each method to gather examples/tips/recommendations and that is now the `package_refernce/<method>` page. This page now also hosts a small space that shows the MetaMathQA (and potentially other) benchmark results highlighted for that method.
I've moved the LoRA initializations to `package_reference/lora#Initialization` and converted the init methods to `<hfoption>`-tags. This collapses them to a list but may reduce searchability through the document - at least firefox is not able to search 'through' the option tabs. This also doesn't make them appear in the ToC and people specifically searching for, say, PiSSA won't find it directly. I think that's OK though, since the search is able to locate it.
The quicktour is a bit more detailed about what happens under the hood (quick doesn't have to mean simplistic) and includes some new visualizations. I hope that we can integrate more visualizations in the future where it makes sense.
* Remove PEFT method space + front page buttons
The space was not that useful anymore since most methods are compatible
with most models.
The front page buttons are, at least temporarily, with the exception
of the quicktour and method overview buttons. I like the visuals
but there should only be elements that are useful.
---------
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Problem description
Right now, if a new PEFT method wants to add support for quantized
layers, it requires a significant amount of work. Notably, the method
needs to implement dedicated layer classes for each quantization method
(e.g. one class for bnb 4bit, one for bnb 8bit, one for AWQ, ...). These
classes typically are >90% boilerplate and the actual difference between
implementations of these classes is minimal.
The result of that is that, at the moment, most PEFT methods don't
support any, or only very few, quantization methods, even though the
amount of actual logic required to support these methods is relatively
small.
Suggested solution
This PR is a suggestion of how to solve the issue. With a few extra
lines, we should be able to support all quantization methods in all PEFT
methods. The general approach is to add an attribute to each PEFT layer,
self.quantization_backend, which supports these methods:
- get_base_weight - set_base_weight
When the PEFT layers use these methods to access and write to the base
layer weight, and if the weight is quantized, the new classes will deal
with that correctly. This means that we no longer need a dedicated layer
class to deal with quantized layers, the normal layer class will do.
E.g. for MiSS, the normal miss.Linear class can deal with bnb layers,
there is no need to add a miss/bnb.py module with dedicated layers.
A few rewrites in the existing PEFT methods are required to support this
new quantization backend class, but the amount of total code needed for
that is considerably smaller than adding new classes for each
quantization method.
Furthermore, these quantization backend classes are agnostic with regard
to the PEFT method. Therefore, with M PEFT methods and N quantization
methods, we no longer need MxN implementations to support quantization
but only M+N.
Migration
For LoRA, we have already implemented the layer classes for each
supported quantization method. For the sake of consistency, it could
still make sense to migrate LoRA to the new approach if it's accepted.
This needs to be accompanied by detailed regression testing to ensure
that everything keeps working. I would only suggest to deprecate and
remove abandoned quantization methods (perhaps for a v1.0 release).
Scope
Updating all PEFT methods is too much for a single PR. This PR focuses
on only three PEFT methods for now:
- MiSS: A pretty normal PEFT method, representative of many other PEFT
methods. - BOFT: Also pretty normal, but requires slight rewrite of the
forward step. Similar changes may be required for other methods too. -
VeRA: Already supports bnb but with this PR, specific BNB layers are no
longer needed.
Implements MonteCLoRA: "Robust and Efficient Fine-tuning of LLMs with
Bayesian Reparameterization of Low-Rank Adaptation"
https://huggingface.co/papers/2411.04358
This LoRA variant augments the LoRA A matrix by including Monte Carlo
estimations of the LoRA parameters. This promises to make the model
more robust to hyper-parameter choice and to lead to better
regularization, at the cost of a small increase in the count of
learnable parameters. There is no extra cost at inference time compared
to normal LoRA.
Implements "VeLoRA: Memory Efficient Training using Rank-1 Sub-Token
Projections"
(https://huggingface.co/papers/2405.17991)
VeLoRA is a LoRA variant that reduces memory required for training by
compressing the activations saved for the LoRA in the forward pass and
then reconstructing them in the backwards pass. Compared to gradient
checkpointing, VeLoRA is faster but requires more memory. The LoRA
adapter itself is unaffected and can be used like any normal LoRA
adapter.
Implements KappaTune based on:
"The Condition Number as a Scale-Invariant Proxy for Information Encoding in
Neural Units"
https://arxiv.org/abs/2506.16289
This is a helper function that can be run on the base model to identify the
layers that are most suited for fine-tuning. It works for Linear and MoE
layers. By targeting these layers with LoRA or other PEFT methods, the model
should train well without forgetting useful information from pre-training.
Now that the PRs in relation to #2864 have been merged, we should also
properly document weight tying behavior in PEFT. I added a section to
the docs based on said issue.
At inference, PEFT can result in a substantial runtime overhead when
targeting MoE parameters. This is now documented, as well as how to
avoid the overhead.
Especially the section on adding new PEFT methods was incomplete and
should be more actionable now.
With the arrival of coding agents, there is even the small chance that
someone might read it.
Torchao made some API changes, which have to be reflected in the tests.
Moreover, for this to pass, we also need transformers to make the
corresponding adjustments:
https://github.com/huggingface/transformers/pull/44604
While working on this, I migrated the tests from unittest to pytest
style.
There's no official rule how we regard typo PRs. To give contributors
a clear vision of how we want typo PRs to be handled I suggest this
update to the guidelines.
Bone was deprecated in favor of MiSS. Removal is scheduled for PEFT
v0.19.0. Old checkpoints can be converted using
scripts/convert-bone-to-miss.py.
Unrelated change: I found the skip for GPT2 for the unloading decoder
test to be unnecessary (it was probably redundant with the skip for
Conv1D layers), so that function was removed completely.
To me the LoftQ application process was not immediately obvious.
I'm sure it can't hurt to add a bit of context what the two main
ways are to apply LoftQ.
In my quest to search for enlightenment I was often directed to the
quantization guide in the PEFT docs which featured a lot less details
than the developer guide for LoRA. Since LoftQ is mainly used in quantization
I think it makes sense to move it there and link there from the developer guide.
Add docs on how to set the rank of LoRA adaptors to the experts in MOEs to be the rank / total experts
---------
Co-authored-by: githubnemo <githubnemo@users.noreply.github.com>
Implementation for https://github.com/huggingface/peft/issues/2907.
After some experimentation I think this is an implementation of the
forgetting through intruder dimensions mitigation presented by the paper
[LoRA vs Full Fine-tuning: An Illusion of Equivalence](https://huggingface.co/papers/2410.21228).
The implementation takes a model with a loaded LoRA adapter and applies the
mitigation to form a new adapter on the same model. This makes it possible
to compare the results directly.
You can find a script to evaluate the changes here: https://gist.github.com/githubnemo/5932e99125d498c4f353017916ffc3ea
Support for `torch.compile` was considered during review but first experiments
didn't yield clear results which is why support was postponed.
## Alternative approaches
I can think of two possible modes how the conversion could be done differently.
The first one is already proposed in the original issue, namely applying the
mitigation on the merged weights. In this case we will lose the ability to
unmerge and modify the adapter directly which loses all the flexibility of
an adapter.
The second way is to modify A/B directly. Since we have the intruder dimensions
and compute the SVD of (W+dW) to build the mitigation vectors it is conceivable
that it also possible to apply the mitigation directly to the adapter's A/B
matrices. Initial testing wasn't successful so I chose the easier (but
computationally more intensive) route of first merging the weights, applying
the mitigation and then re-computing the delta weight (and subsequently the
A/B matrices using SVD of the adapter's rank).
---------
Co-authored-by: nemo <git@ningu.net>
* Bugfix turned into restructuring
Initially I wanted to fix a docs builder bug where the `loraga` autodoc path
was broken since the contents of that module were partially removed.
Then I noticed that LoRA-GA (an initialization method) had its own
`package_resource/lora_ga.md` documentation which is unusual for this kind of
method since as of now all init methods are documented in the LoRA developer guide.
Therefore, I decided to fix the bugs by a) removing the erronous paths and
b) move the package resource page of LoRA-GA to the LoRA developer guide.
Once I arrived there I noticed that aLoRA and DoRA are categorized as initialization
methods which is not true. Also QLoRA and parameter targeting were also in the
same section. These now reside in the new "Training" section. For inference
methods such as aLoRA and Arrow there is now an "Inference" section.
* Small fixes
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Visualize result via table
---------
Co-authored-by: nemo <git@ningu.net>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Resolves#2651
This PR adds caching of the LoRA weight and the weight norm from DoRA
for faster inference. Since, during inference, the weights don't change,
there is no need to recalculate those weights for a DoRA module each
time.
During training, recalculation is needed, thus there is no caching when
the module has training=True.
The cache does not prevent each and every possible duplicate
calculation. For instance, the weight norm is calculated during module
initialization and then again during the first forward pass when
performing inference. Only starting from the second forward pass on will
the weight norms be cached.
The PR includes a script to measure the effect of caching. On my
machine, I get:
avg time LoRA: 0.0717 sec
avg time DoRA no caching: 0.1718 sec
avg time DoRA with caching: 0.0840 sec
memory LoRA: 15612.00 MB
memory DoRA no caching: 16212.00 MB
memory DoRA with caching: 22118.00 MB
DoRA time overhead no caching: 139.52%
DoRA time overhead with caching: 17.08%
DoRA memory overhead no caching: 3.84%
DoRA memory overhead with caching: 41.67%
Thus, caching can significantly reduce inference time but at a
noticeable cost in memory.
Explain how to use multiple adapters (e.g. 2 LoRA adapters) at the same
time, as the API is not quite intuitive and there are some footguns
around trainable parameters.
This question has come up multiple times in the past (for recent
examples, check #2749 and #2756). Thus it's a good idea to properly
document this.
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
This PR adds support for Arrow, a modular routing mechanism for LoRA experts introduced here, as well as the refinement method GenKnowSub, proposed in our ACL 2025 Main Conference paper. GenKnowSub enhances Arrow by subtracting a general-domain LoRA from task-specific ones prior to routing, leading to improved generalisation and modularity.
This PR migrates Activated LoRA (aLoRA) support from a standalone Github (see above) to PEFT itself.
Note there is also an active PR for vLLM inference support for Activated LoRA: vllm-project/vllm#19710 . There are also collections of aLoRA models on huggingface (in the ibm-granite org), note that these preexisting models run off of the standalone github repo and will be updated to work with this new PEFT feature if merged.
Description of changes: Activated LoRA is a modification of the LoRA architecture to "activate" the adapter weights only on tokens coming after a specified invocation_string. This fact makes it so that KV values for the string coming before the activation matches KV values for the base model. This allows KV cache for the input to be interchangeable between the base model and adapter model, and allows for major speedups in inference pipelines (e.g. agentic pipelines) that want to use both base models and adapter models. See the paper for detailed exploration of use cases and further elaboration.
Other notes:
The crux of the changes are really in layer.py. Everything else is simply managing the alora_offsets quantity which defines where the weights start to be activated. This is determined by scanning input strings for the invocation_string defined in the aLoraConfig.
I believe that aLoRA really only makes sense for CausalLMs, hence I've only implemented this for that model type.
Merging doesn't make sense for aLoRA adapters since the weights are not universally applied to all tokens.
I used the LoRA code as a starting point, but did not implement various seemingly extra features in that code.
As of now, invocation_string should probably start and end with special tokens, to avoid tokenizer issues at the boundary. Open to suggestions on how to make this more general if needed.
---------
Co-authored-by: githubnemo <githubnemo@users.noreply.github.com>
There are a few issues with target_parameters that are fixed in this PR.
Existing parametrizations
When using target_parameters with LoRA, after the forward call finishes,
the LoRA parametrization is removed. However, this also used to remove
all other parametrizations on the same parameter, which is bad. With
this PR, only the LoRA parametrization is removed.
Module repr
This PR also extends the __repr__ of lora.ParamWrapper to contain the
parameter name, which makes it more useful.
Extend testing
Added a tiny gpt-oss model to the target_parameters test suite.
Multiple LoRA adapters with target_parameters
There is an issue when adding a second LoRA adapter with
target_paramters, where this second adapter would not actually be
applied correctly. The corresponding unit test was too lax to notice the
bug. This is not easy to fix, so for now we forbid adding a second
adapter with target_parameters. This is very strict but it's better than
having silent errors.
Although it was possible to fix that specific issue, the solution
resulted in ever deeply nested adapters (i.e. with multiple
.base_layer). This in turn results in those infixes to be part of the
state_dict. But then we cannot load the individual adapters correctly,
except if the model is restored in the exact same order as it was
previously created. This is not normally a requirement in PEFT (e.g. I
can create a model with two adapters and later decide to load only one
of them).
In the long run, we need to think about solutions that would allow this.
It may require some form of normalization of the layers to prevent ever
deeper nesting. Also, what is ugly right now is that, given that the
LoRA lives on a module but actually targets one of possibly multiple
parameter, the LoRA weights don't actually reference said parameter in
any name. That means, purely from the state_dict, it is unclear which
parameter a LoRA weight belongs to. Ideally, this should be encoded in
the LoRA weight key.
Make it possible to inject the PEFT adapters based on a state_dict
instead of the PEFT config.
See https://github.com/huggingface/diffusers/issues/11874 for context.
Description
Right now, when creating a PEFT adapter like LoRA, the adapter layers
are injected based on the PEFT config, most notably the entries in
`target_modules`, but other arguments also play into this. Generally,
this is a good approach, but it breaks down in some situations. For
instance, in diffusers, we often have the situation that the checkpoint
was created without PEFT/diffusers, thus there is no PEFT config, only
the `state_dict`. To load these checkpoints in diffusers, the current
approach is to reverse-engineer a valid PEFT config based on the keys in
the `state_dict`.
Unfortunately, this is error prone. Moreover, not every combination of
`state_dict` keys can be easily expressed in a PEFT config through a
combination of `target_modules`, `exclude_modules`, etc. Yes, in theory
everything can be expressed by passing `target_module=<regex_pattern>`,
but reverse-engineering such a regex correctly and efficiently is very
hard (and thus currently not done).
This PR implements a completely different approach to inject adapters.
Instead of relying on the PEFT config to determine which layers to
target, it takes the `state_dict` directly as the source of truth. This
should allow to exactly match what is desired.
Implementation details
I took care to implement this change in a way that if no `state_dict` is
passed, the exact same code path as previously is taken. The risk of
breaking anything should thus be minimized.
Technically, it is not necessary to pass the `state_dict`, we are only
interested in the keys. I still called the argument `state_dict`, since
that is typically what we have at this point, but this can be easily
changed.
I thought it might be a good idea, if the `state_dict` is used, to still
check what modules would have been targeted if we had used the PEFT
config. Then, the results are compared and a warning is given if they
differ. This allows the user to see if the PEFT config is not correctly
specified. While running some diffusers tests, I never encountered this
warning, which is good. However, if we plan, for instance, to get rid of
all the reverse engineering of the PEFT config in diffusers, it would
make more sense to not give this warning.
Caveats
When the original LoRA model was using `target_parameters`, injecting
from `state_dict` will not work correctly. The problem is that the
`state_dict` looks the same, whether the module or a parameter was
targeted. Therefore, we cannot correctly determine the user's intent.
For now, what I decided to do is:
1. Always assume that `target_modules` is meant, as it's the far more
common occurrence.
2. When we detect `target_parameters` while using `state_dict` for
injection, we raise an error.
3. If we don't detect this, injection might just slip through, resulting
in modules being targeted (if they are valid modules) instead of
parameters.
4. Document that these two features don't work together.
I think overall, this is not too concerning, as both features are rather
niche and thus unlikely to be used in conjunction.
Related changes
While working on this PR, I made a couple of related, though not
strictly necessary, changes:
- Refactor tests in `test_low_level_api.py` to use pytest instead of
unittest
- Add default target modules for LoHa and LoKr (just copying LoRA)
- Most PEFT method's model classes like `LoraModel` had an `__init__`
that effectively just called `super()` with the same arguments. I
removed these `__init__` methods.
- Recommends trainable tokens as first measure
- Clarifies a few things about saving embeddings
- Adds full-finetuning as an option of last resort
---------
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>
Normally, nn.Parameter cannot be targeted with LoRA adapters. This can
be problematic, e.g. when there are MoE layers that use nn.Parameter
directly, or when there is nn.Linear but the weight is passed directly
instead of calling forward (e.g. MHA).
It would be possible to craft a solution involving a special LoRA layer
for each of the modules that use nn.Parameter directly (e.g. lora.MHA)
but that doesn't scale. This PR is implements a direct way to target
nn.Parameter making use of torch.nn.utils.parametrize.
Using the feature requires passing target_parameters to the LoraConfig.
During the forward pass, when the parameter is acceessed, the LoRA
weights are added to the weights while still ensuring that gradients
flow correctly to the LoRA weights.
Right now, only LoRA supports this feature. Moreover, it is not possible
to target multiple parameters of the same module with the same adapter.
A workaround is to use multiple adapters (i.e. with different names).
---------
Co-authored-by: githubnemo <githubnemo@users.noreply.github.com>
- Use a more up to date example code in the README
- A section on transformers integration
- Update devs to tag
- Simplify issue template (did not seem useful in practice)
- Update contribution guideline
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
As a user, it should be possible to manually cast the base model to a
lower precision dtype, float16 or bfloat16, and still have the different
PEFT methods work correctly. Currently, this is not the case for many
PEFT methods, as can be replicated by the added tests.
To understand the problem, it helps to take a step back. By default,
PEFT will treat the adapter weights with high precision, i.e. with
float32. When the base model is lower precision, the user needs to pass
inputs in lower precision too, as otherwise self.base_layer(x) would
fail. However, this low precision input clashes with the high precision
adapter weights.
The solution implemented in this PR is to cast the input to a higher
dtype [1]. That way, the whole adapter operation is conducted in high
precision. Only once that has finished will the final result be cast to
the original dtype. This should lead to better results, but it may
require more memory. Note that this is how LoRA is implemented, so the
changes in this PR bring the other methods more in line with what LoRA
does.
If the user does not want the adapter to be in float32, they can always
pass autocast_adapter_dtype=False when calling get_peft_model or
PeftModel.from_pretrained. This is also tested.
Besides adjusting the forward method to account for these changes, the
merge and unmerge methods also often had to be adjusted, as they did not
correctly account for the base model dtype. Now, those methods should
always conserve the original dtype of the base model.
Note that if, for whatever reason, the input casting in [1] is not
desired, users can use the disable_input_dtype_casting context manager
to disable it (more context information on this feature can be found in
PR #2353). I updated the corresponding code to be agnostic to the
specific PEFT method (beforehand, it was only for LoRA).
Note that model.merge_adapter(safe_merge=True) did not work so far, even
though the argument was documented it was not actually there. This is
now fixed.
Supersedes #2382
Right now, the regex used to match the keys passed for rank_pattern and
alpha_pattern requires that either:
1. The module name is identical to the key
2. The module name having a prefix and then ending on the key
This is restrictive, since it doesn't allow to disambiguate between all
cases. E.g. if we have a model with these attributes:
- model.foo
- model.bar.foo
We cannot currently target just model.foo. (We can already target only
model.bar.foo by passing "bar.foo" as a key to the rank_pattern /
alpha_pattern dict).
This PR makes it possible to pass "^foo" as a key. This way,
model.bar.foo is not targeted, as the key does not start with "foo".
As a general rule for users, if they intend to have a full match, they
should pass the full name of the module preceded by a ^. This is the
least ambigious way.
When running the test case with the old code, all the test cases with ^
will fail, which is fine, since ^ was not working anyway. At the same
time, all test cases not using ^ pass, which means they are backwards
compatible.
This is a follow-up PR of #2376 to add support for weight-tying.
Some models, such as gpt2, tie the weights between the LM head and the input embeddings for various reasons. If we use the trainable tokens adapter, we're changing the result of the forward() of the input embeddings but we do not change the weights (unless we merge()). This means that the changes are not reflected in the tied weights, such as the LM head, leading to wrong results when training.
The current approach is searching for tied layers and putting TrainableTokensLayer adapters on them as well but initialized to use the parameters from the embedding layer's TrainableTokensLayer. This is done via the tied_adapter argument of TrailableTokensLayer.__init__().
Notable other changes:
* Implement weight-tying for encoder-decoder models
Notably we are removing the duplication filter of `named_modules` when searching for
the (tied) target modules since tied weights are by definition duplicates.
* Implement embedding name inference
It's now possible to let the adapter decide which is the input embedding layer based on the output
of `model.get_input_embeddings()`. If that fails, the default is still `embed_tokens`.
* Refactor getattr in AuxiliaryTrainingWrapper
Before this change only the selection of the module that was supposed to have the queried
attribute was given to the wrapper implemention (via `_{has,get}attr_wrapped`). Now the full
`getattr()` call is done by the implementation.
This change is motivated by the need for access to `embedding.weight` at certain times which,
for `ModulesToSaveWrapper` is not a problem - but it is for `TrainableTokensWrapper` since
the original module's weights differ from the current weights, at least potentially.
What we do now is to merge the weights and return those when `embedding.weight` is accessed.
No other attributes are currently forwarded.
* initialization from buffers was broken since `persistent` flag was set too late
(update() is called before setting the flag)
* update from other BufferDict was broken since it was assumed that BufferDict was
a mapping collection object. we cannot simply change it to a Mapping since it
then will break pytorch code which assumes that modules are hashable.
---------
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>
This change is based on the nifty addition of @marcusinthesky from #1541.
When adding tokens or fine-tuning the representation of specific tokens we currently have little choice but to retrain the whole embedding matrix which can be huge and adds to the memory footprint (in RAM but also on disk). This method creates a sparse matrix of shape (n, embed_dim) where n is the number of tokens to be customized and only trains these few values.
This change introduces two ways of using it:
```
peft_config = TrainableTokensConfig(target_modules=['embed_tokens'], token_indices=[0, 1, 2])
peft_model = get_peft_model(model, peft_config)
```
and with LoRA
```
peft_config = LoraConfig(
target_modules='all-linear',
trainable_token_indices={'embed_tokens': [0, 1, 2]},
)
peft_model = get_peft_model(model, peft_config)
```
Adding this feature to adapters other than LoRA should be relatively easy, mostly adding the `trainable_token_indices` config option and some debugging.
To make this change it was necessary to change the `modules_to_save` infrastructure as combining this feature with LoRA is quite similar. This refactoring entailed moving most of the basic functionality of `ModulesToSave` to the `AuxiliaryTrainingWrapper` class. This also changes the logic how `modules_to_save` is loaded/saved from from the state dict, so there could still be bugs here.
This implementation does not entail support for weight-tied layers yet. This will follow in a future change.
---
Notable commits in this squash:
* Use unload_and_optionally_merge_module protocol
With `AuxiliaryTrainingWrapper` as abstraction it is probably a good idea to
have support for `unload_and_optionally_merge_module`.
Since the wrapper is more akin to a PEFT layer than a model the name semantics
are fine and it does basically the same job.
* trainable tokens is also trained in certain adapters
Before, the assumption was that modules_to_save was the only thing that
is trained alongside an adapter's parameters. Now there's also the
token_adapter delta tokens via `NewTokensWrapper`.
* Remove old modules_to_save handling
This is now all handled via the `AuxiliaryTrainingWrapper`.
* Fix modules_to_save module overwriting
The state dict imlementation of ModulesToSaveWrapper was incorrect in that
it did not include its own parameters, just the parameters it needs to overwrite
in the end. I.e. if layer `lin1` is modules to save wrapped,
`lin1.{weight,bias}` is saved and overwritten but `lin1.modules_to_save.<adpater_name>.[...]`
is not saved.
* Introduce a load key map for aux. train wrapper
Before this change it was only possible to remove a key prefix from the wrapper's
state dict (e.g., `modules_to_save.default.weight` -> `weight`); now it is possible
to restore such reduced value by mapping the key back
(i.e., `weight` -> `modules_to_save.default.weight`).
* Replace sparse matrix with dense + index_copy
This change is mostly because sparse matrices are not that beneficial in this case
(at least not from what we can see right now) and they do not solve the problem
of having to change the new tokens in-place to avoid outdated deltas when new token
vectors are initialized randomly after loading the deltas.
* Make peft_config.layers_to_transform optional
Before this change the base tuner class was forcing this attribute
to be present on the config class even though the attribute is not
specified in the base config.
* Implement missing key logic in `_set_trainable`
Before this it was not checked if the targeted module by `modules_to_save` or `trainable_token_indices` existed
or not (when used in conjunction with a PEFT method). In this case an error message similar to the `inject_adapter`
error is raised when no module is found.
---------
Co-authored-by: Marcus Gawronsky <marcus.g@myrunway.co.za>
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>
Users sometimes get confused by the warning from transformers that some
weights are uninitialized and need to be trained when they use models
for classification. A recent example is #2367.
Even though the warning does not come from PEFT, let's add a section to
the docs to explain this warning, as the situation is a bit different
here.
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
We have tests to check if torch.compile works for various PEFT methods
and "advanced" features (QLoRA, merging, ...). These tests are not run
on a regular basis, but are triggered manually. As such, it was time to
revisit them.
So far, a few of these tests were marked as xfailing. All these tests
are passing now. The reasons for this:
- Presumably: New PyTorch version (I haven't checked older)
- Loosening some tolerances
- Remove a spurious argument added by torch.compile
- Slightly adjust order of when torch.compile is called
The docs have been updated to reflect these new findings.
There have been multiple issues and forum posts in the past asking about
errors like:
TypeError: LoraConfig.__init__() got an unexpected keyword argument ...
This error can occur when the adapter that is being loaded is trained
with a more recent PEFT version than the one currently being used. I
thus added a section to the Troubleshooting part of our docs to describe
the solutions.
Note that we already added changes to PEFT in #2038 to make configs
forward compatible. But since users who encounter this problem have, by
definition, older PEFT versions, they don't benefit from this.
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Implements the paper "CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning" (https://arxiv.org/abs/2406.05223)
This initialization method can be used for building task-aware LoRA adapters from weight decomposition oriented by the context of the task using examples from data.
---------
Co-authored-by: 5eqn <491100866@qq.com>
Ihor Vitenko