Remove Azure Secrets committed to GitHub

Writing code day after day means secrets, connection strings and more get added to your code. And if you are like me, they get committed to your GitHub database and then you are embarrassed. In this post, I'll walk you through cleaning up a repo.

Part 1:

  • Change to the directory where you store your repo or clone a fresh copy with git clone https://github.com/something/something.git.
  • Clone a fresh copy of your repo using the mirror option, like the following git clone --mirror https://github.com/something/something.git.
  • You'll now have a bare repo. Below I've listed out the contents to verify.
Michaels-MBP:cleanme mbcrump$ ls -l
total 16
-rw-r--r--   1 mbcrump  staff  189 Apr 16 20:25 appsecrets.json
drwxr-xr-x  11 mbcrump  staff  352 Apr 16 20:26 cleanme.git
1
2
3
4
  • You see that I have an appsecrets.json which contains some sensitive data that I need to remove:
{
  "ConnectionStrings": {
    "StorageAccountAPI": "DefaultEndpointsProtocol=https;AccountName=autotweet;AccountKey=+1234;EndpointSuffix=core.windows.net"
  }
}
1
2
3
4
5

Part 2:

  • Install BFG with brew install bfg assuming you have Homebrew installed and using a Mac or download the JAR file if you are on Windows.

Part 3:

  • Build a passwords.txt file and place and enter the passwords that you'd like to remove.

Mine was replacing an Azure Storage Table key that I accidentally committed:

DefaultEndpointsProtocol=https;AccountName=autotweet;AccountKey=+1234;EndpointSuffix=core.windows.net
1

Part 4:

  • Run bfg --replace-text passwords.txt cleanme.git
  • Below is output from that command:
Cleaning
--------

Found 7 commits
Cleaning commits:       100% (7/7)
Cleaning commits completed in 253 ms.

Updating 1 Ref
--------------

	Ref                 Before     After   
	---------------------------------------
	refs/heads/master | 1aa1546d | 1aac2fd2

Updating references:    100% (1/1)
...Ref update completed in 89 ms.

Commit Tree-Dirt History
------------------------

	Earliest      Latest
	|                  |
	 .  D  m D  D  D  m 

	D = dirty commits (file tree fixed)
	m = modified commits (commit message or parents changed)
	. = clean commits (no changes to file tree)

	                        Before     After   
	-------------------------------------------
	First modified commit | 2b978e2d | 82b385c5
	Last dirty commit     | be1cde85 | 8ddbdf06

Changed files
-------------

	Filename          Before & After     
	-------------------------------------
	appsecrets.json | 5d8bfd89 ⇒ 21cdaeb1


In total, 10 object ids were changed. Full details are logged here:

	/Users/mbcrump/Documents/GitHub/cleanme/cleanme.git.bfg-report/2018-04-16/21-20-48
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

Part 5:

  • Run git reflog expire --expire=now --all && git gc --prune=now --aggressive
  • Run git push to push it to your repo.