This is a text-only version of the following page on https://raymii.org:
---
Title       :   Git-clean
Author      :   Remy van Elst
Date        :   08-11-2012
URL         :   https://raymii.org/s/software/Git-clean.html
Format      :   Markdown/HTML
---



The script below is a script to clean a git repository. Either to see big files,
or to remove them, and remove them from history.

<p class="ad"> <b>Recently I removed all Google Ads from this site due to their invasive tracking, as well as Google Analytics. Please, if you found this content useful, consider a small donation using any of the options below:</b><br><br> <a href="https://leafnode.nl">I'm developing an open source monitoring app called  Leaf Node Monitoring, for windows, linux & android. Go check it out!</a><br><br> <a href="https://github.com/sponsors/RaymiiOrg/">Consider sponsoring me on Github. It means the world to me if you show your appreciation and you'll help pay the server costs.</a><br><br> <a href="https://www.digitalocean.com/?refcode=7435ae6b8212">You can also sponsor me by getting a Digital Ocean VPS. With this referral link you'll get $100 credit for 60 days. </a><br><br> </p>


#### Examples

View the list of big files in the git repo and the git repo history (from the
sourcecode repo of this website):



   remy at sparcstation-20 in ~/repo/raymiiorg (master)
   $ ~/bin/git-clean.sh df
   All sizes are in kB's. The pack column is the size of the object, compressed, inside the pack file.
   size     pack    SHA                                       location
   2051757  655294  7e94729f780724d3b00803b018199f137959630e  includes/sec/dutch.txt
   901683   901853  52e346e5ce9a2f4d9d531f93c082178684da4c72  images/smalldistro12.png
   830998   827833  9afcd85c73a994ce1dc1e5daf1856113d6bf85f0  images/smalldistro10.png
   799361   765804  a8e7d464fbb7e5159442b2644a4f60f6352bf238  content/downloads/packages/gource-0.38-2.x86_64.rpm
   793898   763445  ea80fcf775a070e8d7c2b9375316f27cdac45bf6  content/downloads/packages/gource_0.38-1_amd64.deb
   753400   746864  d9901f182cbd7090efe679847851027e1e05882c  content/downloads/packages/logstalgia-1.0.3-2.x86_64.rpm
   747480   743285  e5aa2be4d510c14c1e67f2c07dbcf812a95a0798  content/downloads/packages/logstalgia_1.0.3-1_amd64.deb
   635854   636023  6e5c39dff693b861f25ec96ba16c5fe9700ec9c9  images/smalldistro3.png
   443121   439081  4461b7489dcb46911e14376e37e7abbe335668a7  images/smalldistro2.png
   395247   393304  47a6ab134240e2433310cc40d8909bd11cf80935  images/smalldistro7.png


Remove the file images/smalldistro12.png from the repo and rewrite the history
(note the backslash in the filename):



   remy at sparcstation-20 in ~/repo/raymiiorg (master)
   $ ~/bin/git-clean.sh rm imagessmalldistro12.png
   Removing Biggest file imagessmalldistro12.png from git repo
   Rewrite 2f65a614583053c57934b1b9ad378b77ac1ddbb9 (259/259)
   WARNING: Ref 'refs/heads/master' is unchanged
   WARNING: Ref 'refs/remotes/origin/master' is unchanged
   WARNING: Ref 'refs/remotes/origin/master' is unchanged
   WARNING: Ref 'refs/stash' is unchanged
   Counting objects: 2159, done.
   Delta compression using up to 4 threads.
   Compressing objects: 100% (785/785), done.
   Writing objects: 100% (2159/2159), done.
   Total 2159 (delta 1366), reused 2152 (delta 1363)
   Counting objects: 2159, done.
   Delta compression using up to 4 threads.
   Compressing objects: 100% (2148/2148), done.
   Writing objects: 100% (2159/2159), done.
   Total 2159 (delta 1368), reused 791 (delta 0)


#### Info

##### Installation

Copy the script into a text file, `chmod +x ./git-clean.sh` and run it. If you
do not run it from inside a git repository you will get an error.

##### License

GPLv3.

#### The script



   #!/bin/bash
   # Script to remove files from the git history.
   # Execute this script in the git root directory
   # For a size report: ./gitsize.sh df
   # To remove the big files:
   # use like: ./gitsize.sh rm $bigfilename
   # Example: ./gitsize.sh rm includes/ubuntu-12.04.iso
   # It will then search your git repo and remove all of it.

   # Copyright (C) 2012 Remy van Elst

   #     This program is free software: you can redistribute it and/or modify
   #     it under the terms of the GNU General Public License as published by
   #     the Free Software Foundation, either version 3 of the License, or
   #     (at your option) any later version.

   #     This program is distributed in the hope that it will be useful,
   #     but WITHOUT ANY WARRANTY; without even the implied warranty of
   #     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
   #     GNU General Public License for more details.

   #     You should have received a copy of the GNU General Public License
   #     along with this program.  If not, see <http://www.gnu.org/licenses/>.

   if [ ! -e $1 ]; then
       if [ -d .git ]; then
           if [ $1 == "rm" ]; then
               if [ ! -e $2 ]; then
                   echo "Removing Biggest file $2 from git repo"
                   DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &&&& pwd )"
                   rm -rf .git/refs/original/
                   git filter-branch --index-filter "git rm --cached --ignore-unmatch $2" --tag-name-filter cat -- --all
                   rm -rf .git/refs/original/
                   git reflog expire --all --expire-unreachable=0
                   git repack -A -d
                   git prune
                   git gc --aggressive
               else
                   echo "I need a file to remove..."
               fi
           elif [ $1 == "df" ]; then
               IFS=$'n';
               echo "All sizes are in kB's. The pack column is the size of the object, compressed, inside the pack file."
               objects=`git verify-pack -v .git/objects/pack/pack-*.idx | grep -v chain | grep -v pack | sort -k3nr | head -n 10`
               output="size,pack,SHA,location"
               for y in $objects
               do
                   # extract the size in bytes
                   size=$((`echo $y | cut -f 5 -d ' '`))
                   # extract the compressed size in bytes
                   compressedSize=$((`echo $y | cut -f 6 -d ' '`))
                   # extract the SHA
                   sha=`echo $y | cut -f 1 -d ' '`
                   # find the objects location in the repository tree
                   other=`git rev-list --all --objects | grep $sha`
                   #lineBreak=`echo -e "n"`
                   output="${output}n${size},${compressedSize},${other}"
               done

               echo -e $output | column -t -s ', '
           fi
       else
           echo "Cannot find .git directory, exitting. This script should be ran from a git working dir."
           exit 1
       fi
   else
       echo "Usage:"
       echo "    For a size report:"
       echo "    $0 df"
       echo "    "
       echo "    To remove a big file from git history:"
       echo "    $0 rm <filename>"
       echo "    Example, to remove the file "includes\ubuntu-12.04.iso":"
       echo "    $0 rm includes\ubuntu-12.04.iso"
       echo "    Made by Raymii.org. GPLv3 License"

   fi


  [1]: https://www.digitalocean.com/?refcode=7435ae6b8212

---

License:
All the text on this website is free as in freedom unless stated otherwise.
This means you can use it in any way you want, you can copy it, change it
the way you like and republish it, as long as you release the (modified)
content under the same license to give others the same freedoms you've got
and place my name and a link to this site with the article as source.

This site uses Google Analytics for statistics and Google Adwords for
advertisements. You are tracked and Google knows everything about you.
Use an adblocker like ublock-origin if you don't want it.

All the code on this website is licensed under the GNU GPL v3 license
unless already licensed under a license which does not allows this form
of licensing or if another license is stated on that page / in that software:

   This program is free software: you can redistribute it and/or modify
   it under the terms of the GNU General Public License as published by
   the Free Software Foundation, either version 3 of the License, or
   (at your option) any later version.

   This program is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
   GNU General Public License for more details.

   You should have received a copy of the GNU General Public License
   along with this program.  If not, see <http://www.gnu.org/licenses/>.

Just to be clear, the information on this website is for meant for educational
purposes and you use it at your own risk. I do not take responsibility if you
screw something up. Use common sense, do not 'rm -rf /' as root for example.
If you have any questions then do not hesitate to contact me.

See https://raymii.org/s/static/About.html for details.

You are viewing proxied material from raymii.org. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.