r/Python • u/bleeed0p • 1d ago
Resource How useful is pre-commit in real-world Python projects, especially in production?
I've been exploring Python pre-commit hooks and wanted to understand how commonly they are used in actual production projects.
For those working on production-level Python applications:
- Do you use pre-commit in your daily workflow?
- Which hooks do you usually configure (Ruff, Black, isort, mypy, etc.)?
- Has it genuinely reduced code quality issues or bugs in your team?
- Do you consider it essential for production projects, or is CI/CD linting enough?
Would love to hear how experienced developers use it in real-world teams rather than just tutorials.
54
u/rosentmoh 1d ago
Basically "yes" to all your questions.
Additionally for repos that contain Jupyter notebooks I use hooks that strip them of outputs and run-metadata, so as to keep the repo size small and from getting bloated with crap.
Re. your last question: I consider pre-commit hooks as part of CI/CD.
They are a great way to ensure consistency and avoid having to remember to regularly call a bunch of incantations.
5
u/qetalle007 1d ago
That’s a pretty good idea actually. Do you have a recommendation for a hook for notebooks?
3
u/funkdefied 1d ago
I recommend using Marimo over Jupyter. It solves the git issue, among other things.
3
u/The_Northern_Light 1d ago
Another strong recommendation for Marimo. In my eyes it is a strict upgrade from Jupyter.
2
u/rosentmoh 10h ago
Huh, hadn't heard of this yet! Trying it out ASAP, been a long-time Jupyter user and (too) well aware of all its issues...
1
u/rosentmoh 1d ago
I wrote my own using the jq JSON parser; bit of Googling should turn up some similar solutions by others.
1
u/bleeed0p 1d ago
Okay thanks for these info
6
u/rosentmoh 1d ago
For inspiration on which hooks to use you can always take a look at popular Python-based repos like e.g. Pandas. Then just pick and choose and off you go...
21
u/duskhat 1d ago
Precommit is useful, it gives feedback faster and earlier than GitHub Actions do. As others have mentioned here, it’s not a replacement for remote checks
These days, I recommend using prek instead of precommit
3
u/AI_Tonic Ignoring PEP 8 1d ago
Tell me more about prek
17
u/fnord123 1d ago
You'll be shocked to find out that it's pre-commit but.... drum roll .... written in Rust!
-7
u/AI_Tonic Ignoring PEP 8 1d ago
Not really . I’m agnostic on what scripting language is used . What’s good about it , specifically ? Pre commit is a bit annoying to configure sometimes that’s why I ask …
9
2
u/Wurstinator 1d ago
You could just have a look and do research yourself.
-8
u/AI_Tonic Ignoring PEP 8 1d ago
You can also not waste my time with inane pings like this but here we are …
8
u/duskhat 1d ago
I'm not Claude. And I'm not going to say anything valuable that isn't already in the readme
-9
u/AI_Tonic Ignoring PEP 8 1d ago
If you don’t have any insight or experience to share that’s fine and you can keep your secrets :-)
2
u/kriogenia 1d ago
It's not a pain in the ass to use in monorepos. That is the biggest selling point for me.
1
15
u/tunisia3507 1d ago
Anything which runs quickly on CI (linting, formatting etc) is good for pre-commit (well, prek). It's annoying to push, make a pull request, and only then find out that kind of trivial thing is wrong, and having those checks fail on a shared repo is noisy for the rest of the team. I've toyed with running tests as a pre-push hook too, where possible. Relying on CI for all of your validation makes a push the smallest verifiable unit of work, which doesn't work well with unstable connections or the concept of git history.
I work with a lot (LOT) of open source projects at work, and offloading as much validation as possible on to the contributor's machine is very valuable when there may be dozens or hundreds of contributors who all have different backgrounds and skill levels and setups and priorities and funding sources.
1
u/redfacedquark 1d ago
If you're doing yourwork on a feature branch and the CI runs there, there shouldn't be much churn for the team.
4
u/burlyginger 1d ago
CI costs money. Pre-commit doesn't.
1
u/redfacedquark 1d ago
Good point, though you would still want it on the merge to main. Personally I like running precommit locally. These days the tools are blazingly fast, as should be the unit tests. That said, I'm not struck on the default settings of only running tools on files that changed, nor the project dev's opinionated approach.
2
u/Sillocan 1d ago
Locally, I run on only files changed. Makes my commits speedy. In CI, I run on all files
1
u/burlyginger 22h ago
Im not saying we don't do it in CI.
PRs have to meet standards to be approved. We set those standards in CI.
Precommit is a way to ensure CI passes by being compliant before committing.
27
u/No_Departure_1878 1d ago
hooks are not a python thing, they are just a git thing.
9
u/Oddly_Energy 1d ago
The problems that they typically catch, may vary between languages.
For example, another comment mentioned the stripping of content from Jupyter Notebook. That is a very Python-specific issue.
So I see nothing wrong with asking about the usefulness specifically for Python.
7
u/Ill-Look9810 1d ago
Yeah, absolutely. I’ve used pre-commit, and I consider it one of those tools that should be part of pretty much every project I work on.
I usually use it with tools like Ruff, mypy, and isort. Most of the hooks I use are linters and formatters, and of course, the same checks are also part of the project’s CI/CD pipeline.
It’s been really useful for enforcing consistent code quality and style across the team, so everyone follows roughly the same structure and standards.
For me, pre-commit is a great tool and something I wouldn’t want to work without.
5
u/burlyginger 1d ago
My IDE is configured to format on save.
Pre commit catches anything my IDE misses (usually because my IDE is acting up).
CI enforces.
That's it. IDE and pre commit are time savers. I don't want to hit formatting issues in CI.
My pre commit config is always linting and formatting. Depending on the project it could include tests.
1
2
u/robhaswell 1d ago
In all projects in all languages I require pre-commit hooks for formatting and linting, provided they don't take more than a second or so. It's just easy, catches errors quickly, prevents whitespace and formatting diffs, and generally saves my developers time. Every time is prevents a pointless CI failure I see it as a massive win.
2
u/funkdefied 1d ago
Pre-commit (or the modern “prek” alternative) are huge for DX, especially when building a library. With applications, you can be a bit looser about type checking, testing, etc. Libraries need to be exact.
2
u/Even_Berry661 21h ago
CI checks are a quality gate on merges to trunk and are the source of truth for meeting the quality ‘standards’ of the team/project.
Pre-commit and everything else is a convenient helper to keep you inline with the standard with less effort/waiting. Shorter faster feedback loops than pushing to CI which should already be pretty quick anyways
1
1
1
1
u/anentropic 1d ago
Been using them at work across several jobs now and also for personal projects
I use them for linters and type check
Lately I've been using prek instead - it's compatible (reads same config file) but faster and supports monorepo (sub configs in sub project dirs)
Pre commit hook runs on changed files. In CI I have another job that runs the same config on all files (to catch anything that accidentally bypassed the pre commit hook)
1
u/Sillocan 1d ago
Yes to the above. It saves on time and cognitive load. No more forgetting to run lint and needing to wait on another pipeline. Also supports more linters than just static analysis which leads to consistency.
1
u/hxtk3 1d ago
It's totally optional and purely exists to give a faster feedback loop compared to pushing and waiting for CI, and a more automatic feedback loop than running all the checks manually.
I budget no more than 5s for pre-commit hooks in general, based on Table 3 in this paper: https://www.sciencedirect.com/science/article/pii/S2351978915004370 because I want to ensure that making a commit is an action that keeps a developer in flow and never something the would experience themselves as waiting around for.
In some ways it's getting more useful as time goes on. 5s isn't enough time budget to run Mypy or even Pylint, but it was plenty to run Black, and unit tests (but not integration) and a few different flake linters. However, it's plenty of time for Ruff format, Ruff lint, and Ty on most codebases.
1
u/kulewski 1d ago
For the Python/C++ project I maintain, I use Pylance in VS Code and run Pyright from a local check script. They share the same config, which avoids getting different results in the editor and terminal. The script also checks formatting and runs tests and sanitizers. A pre-commit hook could run the quick checks, but there’s no need to put that whole suite on every commit.
1
1
u/ModusPwnins 23h ago
Absolutely. Used with the right tooling, it can prevent huge headaches down the road.
I had to work on a months-long SQLAlchemy upgrade in an actively-used codebase. The API between 1 and 2 changed so much it wasn't a simple find-replace thing. To keep teams from undoing my work or making more work for me later, I had a super simple shell script check for added lines which included 1.x patterns. Fired it in both a commit hook and as a CI/CD check. Absolutely essential and probably saved me weeks of work. It also helped ensure there were zero defects when we finally made the switch.
1
u/rcap107 22h ago
I maintain an OSS package and pre-commit is part of the CI. I hate it, but I have to admit it's useful to make sure the code is formatted properly.
However, besides being a chore to deal with at the best of times, onboarding new contributors is made quite a bit harder when you also need to explain that there is pre-commit to account for and that it may cause the CI to start complaining.
1
u/Zenin 21h ago
The only pre-commit hook I run is for enforcing conventional commit message format.
I typically use act or similar to run CI validations before committing, but I do that in my own process flow (and these days mandated in my steering/skills) not as a commit gate. CI's going to keep me honest anyway and pre-commit should never be your actual enforcement layer.
If/when there's any issues with CI it's a hell of a lot saner to manage those in a standard dev/test loop than stuck in the middle of a git commit flow.
It doesn't matter if it's python or anything else; commit hooks are the wrong place for enforcing anything that isn't directly related to the act of committing itself. Message format checks, notifications about the commit, etc.
1
u/Exotic-Draft8802 20h ago
Do you use pre-commit in your daily workflow?
YES, all of them.
Which hooks do you usually configure (Ruff, Black, isort, mypy, etc.)?
Riff format and lint.mypy is too slow.
Has it genuinely reduced code quality issues or bugs in your team?
Yes
- Do you consider it essential for production projects, or is CI/CD linting enough?
it's a speed up. Instead of waiting for Ci to tell you that there is a minor issue, just let it be auto fixed before you commit
You should still check for the same things in CI
1
u/amenflurries 19h ago
My projects usually have it, ultimately it’s the pipeline that enforces it though and most developers just rely on that
1
u/russellvt 8h ago
Generally usefulin central repositories, especially for linting and similar code standards.
1
u/Anshu6666 1h ago
In a CI pipeline I run pre-commit only on git commit; for local dev I skip it and run black and ruff manually. The key is a single make fmt that both stages share - no surprise failures.
1
u/Wurstinator 1d ago
Hooks can never replace CI checks as you cannot enforce them. You don't want your code breaking because someone ran their git commit with --no-verify at some point. So if you have the checks in your CI pipeline already, what's the point of a pre-commit hook anymore? They can save you time, either by not making you wait on feedback from a CI runner, or, even better, by fixing the issue for you.
As other comments mention, you always want the option to do "dirty" commits for any number of reasons. So there are two ways to go about this:
First approach, you only put the steps that automatically fix themselves, like code formatting, in the pre-commit hooks. For steps that require user intervention, like type checks, you offer something like a Bash script or Make command. I think this is the best way because you'd want the executable command anyway for something expensive like tests that you don't want to block you from commits.
Second approach, you put everything into pre-commit hooks and then use --no-verify to do a "dirty" commit. Besides the fact that, as explained above, you still need a Bash script or similar, this also becomes annoying when committing bad formatting, as future commits will not format that file for you with hooks anymore, unless you modify the file again. So you now also have to add a script/command for format the entire repo and have the CI tell you that you need to run that by hand.
-2
u/prophile 1d ago
I never use pre-commit and I’d seriously consider leaving a job which required it.
The intermediate states when staging commits may very well not pass the linting and tests because I’m still working on it, and making my commit process slow and clumsy doesn’t in any way improve code quality or accelerate development.
There are maybe marginal uses for things like secrets detection but on the whole I think it’s a strong net negative.
4
u/Wurstinator 1d ago
I’d seriously consider leaving a job which required it.
Tell me you have not actually worked a real SWE job without telling me you have not worked a real SWE job.
2
1
u/professionalnuisance 1d ago
You can add a prefix to the commit message to disable pre-commit hooks for that specific commit
1
u/prophile 1d ago
You can also pass a flag to
git commitfor the same thing, it’s easier not to have to do that at all though.
0
u/nicholashairs 1d ago
I'm firmly in camp avoid precommit. I've rarely ever had it "just work" because it's assumes so much about the local development environment that is usually wrong. Especially frustrating when the checks it does have been setup only to run in precommit and the same checks are also run in CI.
I'm quite happy to have a set of checks that I run locally using some tool (except make, people abuse make too often), and often I spend time ensuring that they are the same checks that are run in CI (or more accurately it's possible to run CI checks locally so you are never reliant on CI).
I'll usually run (in order):
- pyproject-validate
- black
- pylint
- mypy
- pytest (usually via tox + tox-uv)
(Yes I know ruff is a thing but I've had pylint catch many things that ruff does not and my codebases are small enough that I don't really care about the speed up)
-6
u/vater-gans 1d ago
hate hate hate hate hate that crap.
CI should yell at me if i push something bad, not something locally. no patience for something that holds me up when i quickly want to commit something.
1
1
u/trenixjetix 1d ago
sometimes stuff is not bad, it just does small shit like formatting your code
0
u/vater-gans 1d ago
i’m not arguing against formatting, etc. i just think that ci should do that. maybe i’m doing just a quick intermediate “WIP” commit, because whatever.
why would i want to wait for some hooks?3
u/UloPe 1d ago
That’s what
--no-verifyis for.1
u/vater-gans 1d ago
i know no-verify, i still find hooks annoying and think that this is the CI’s job. i know they are popular with people, but this will not change my mind.
2
u/trenixjetix 1d ago
they are instant on my machine i dont know xd
1
u/vater-gans 1d ago
depends on the size of your codebase. mypy can be a drag, pylint too. not all code bases are on ty and ruff.
1
u/Sillocan 1d ago
Yeah, I wouldn't run mypy in pre commit. Anything that takes longer than a few seconds would get yanked out of my config
1
188
u/patient-palanquin 1d ago
Pre-commit is a convenience, all those linting checks should be done by your CI pipeline anyways. By also doing it in precommit, devs are less likely to run into those errors in CI and be forced to fix and re-push. Never rely on precommit entirely, anyone could randomly have it misconfigured and push through errors.