How to find and export your GitHub data for AI training

01

What is GitHub data, and why do AI labs and data vendors buy it?

What a GitHub organization records

GitHub is a service that hosts Git repositories. Git is the tool most software teams use to save versions of their code, and a repository is a project's folder together with the complete history of every change made to it. A company's GitHub organization is its account, and it holds all of the company's repositories.

Commit

One saved set of changes, recorded with its author, its time and a short message that says why the change was made.

Branch

A separate line of work, where a developer can try a change without touching the main version of the code.

Pull request

A proposal to merge a branch into the main code, where colleagues read the change, comment on it and approve it.

Issue

A written report of a bug or a request for a feature, with a discussion beneath it.

Why a repository's history teaches an AI model something

An AI model learns by studying a very large number of examples and finding the patterns in them, and those examples are called training data. The finished code in a public project shows only the end result. A company's repository history shows how the code changed: the problem someone reported, the change that was proposed, the objection a reviewer raised and the fix that followed.

AI labs now want to train AI agents, software that carries out a job step by step, and writing and repairing code is one of the jobs they care about most. Company repositories also hold what public projects often lack, such as the internal frameworks a business built for itself, the tests written after real failures, and the database changes that show how a system grew over years. The full mechanism is in AI training data: why AI companies want your company's data.

How GitHub fits into a dataset

GitHub is one of several apps in a company's dataset, which usually also includes chat, email and project-tracking records. Companies are paid $100K to $2M or more per dataset, according to micro1, the data vendor we typically refer companies to. The price covers the whole dataset, not each app on its own, and what your data is worth explains what moves it.

02

Where does GitHub show how much data you have?

How to measure your repositories

The steps below build the total in gigabytes (GB) from the organization's repositories. A mirror clone is a complete copy of a repository's Git data, including every branch and tag, and it is the closest local match to what GitHub stores.

1. As an organization owner, open the organization's page and choose the Repositories tab to see all of its repositories.
2. A quick inventory is available from the REST API by sending GET /orgs/ORG/repos with a token that is allowed to read private repositories. Every repository in the reply has a size field, which is a rough guide, while the clone in the next step gives the figure to report.
3. The exact size comes from a mirror clone of each repository, followed by a count of its objects, as in the commands below. On its size-pack line, Git prints the compressed size of the whole history.
4. A repository that keeps large files in Git LFS needs one more command, git lfs fetch --all, run inside the mirror, and the size of the files it downloads is added to the total. Git LFS (Large File Storage) stores big files, such as images or datasets, outside the normal Git history.
5. Sum the figures from every repository and express the total in GB.

git clone --mirror \
  https://github.com/ORG/REPO.git
cd REPO.git
git count-objects -vH

GitHub's repository limits page defines on-disk size as the size of the compressed .git folder, which is what a mirror clone contains.

Who can see the full count

Organization owners can see and clone every repository in the organization. A member with read access can measure only the repositories that member can read, so a figure from a member's account may miss private repositories.

Why the total leaves out issues and pull requests

Cloning copies only what Git itself stores. Issues, pull requests and review comments live in GitHub, so their volume is a separate number that you note beside the code size if you plan to include them.

03

What should you know about your GitHub data before you apply?

The size in GB

Report the combined size-pack of your mirror clones in GB, and include Git LFS objects if you use them. If you plan to license issues, pull requests or review comments too, say so, because their volume is separate from the size of the code.

The years of history

The years of history come from the date of the oldest commit across your main repositories. Running git log --reverse on the default branch lists the oldest commit first.

The number of members

Count the members of the organization, not the outside collaborators. Outside collaborators are people who can reach a repository without belonging to the organization, such as a contractor.

What you don't need to do first

You don't need to export anything to apply, and you don't need to clean or scrub a repository. The data vendor anonymizes the records before anyone else sees them, and nothing moves until a lawyer-reviewed license is signed. The application, the intro call and each later step are laid out in how to sell your company's data.

04

How do you export your full GitHub history?

Step 1: copy the code and its history

1. Run git clone --mirror with the URL of each repository, which copies all of its branches, tags and commits.
2. A repository that relies on Git LFS also needs its large files brought in, which git lfs fetch --all does when run in the mirror.
3. Compress the folder into a .zip or .tar.gz file. GitHub's guide to backing up a repository describes the same sequence.
4. Clone each wiki the same way, because a GitHub wiki is itself a Git repository.

Step 2: export issues, pull requests and comments

Only an organization owner can run the organization migrations API, and it needs a classic personal access token, since GitHub's API reference says these endpoints do not work with fine-grained tokens or GitHub App tokens.

1. Start the migration with POST /orgs/ORG/migrations, sending the request body shown below, which lists the repositories to include and is sent with the token and an Accept: application/vnd.github+json header. The lock_repositories option defaults to false, which leaves the repositories usable while the export runs.
2. Check the migration with GET /orgs/ORG/migrations/ID until its state reads exported. The other states are pending, exporting and failed.
3. Fetch the archive from GET /orgs/ORG/migrations/ID/archive, which redirects to the file. It arrives as a compressed tar.gz package holding JSON records, attachments and the Git data.
4. Save the file within seven days, after which GitHub deletes it. You can delete it sooner.

POST /orgs/ORG/migrations
{
  "repositories": [
    "ORG/REPO-ONE",
    "ORG/REPO-TWO"
  ]
}

The options exclude_attachments and exclude_releases shrink the archive, and leaving them off gives the most complete copy. An owner of a personal account can export the repositories that account owns the same way with the user migrations API.

What each method captures

The two methods overlap on the Git data and differ on everything around it. The diagram shows the same four kinds of content for each.

What GitHub's export leaves out

A mirror clone does not hold release files, which live outside Git. A migration archive adds JSON records of issues, pull requests, releases, repositories and users, along with attachment files, yet GitHub states that Git LFS objects, discussions and packages are not part of it. There is also no supported way to restore an archive on GitHub, so treat it as a record to keep, not a backup to restore from.

Size limits that shape how you export

GitHub recommends keeping a repository under 10 GB on disk, blocks single files over 100 MB and limits a push to 2 GB, as its repository limits page lists. That is why large binary files usually sit in Git LFS, and GitHub's storage and bandwidth page lists how much LFS storage each plan includes: 10 GiB on Free and Pro, and 250 GiB on Team and Enterprise Cloud.

Customers of GitHub Enterprise Server, the version a company runs on its own servers, can also move data out of their own instance with a tool called ghe-migrator.

Sources: GitHub Docs, Backing up a repository, REST API endpoints for organization migrations and Repository limits. Steps checked against these pages in November 2026.

05

What stays private when GitHub data is licensed?

You choose what is in scope

A company picks the repositories it wants to license, so repositories for infrastructure, security work or experiments can stay out. A repository's history can also hold things nobody meant to keep, so the data vendor deletes records too sensitive to clean safely instead of anonymizing them.

Names are replaced before a lab sees them

Anonymization means replacing every detail that identifies a person, such as a name or an email address, with a stand-in that is the same every time it appears. In a repository, the details are mainly the author names and email addresses on commits and the names in pull request comments. The sequence of the change, the review and the fix still reads in order, which is what a lab learns from. The method is in how company data is anonymized for AI training.

Ownership and payment

Licensing gives the buyer limited rights to a copy, while the company keeps ownership of the originals. The terms that matter, such as exclusivity, permitted use and deletion, are covered in data licensing. micro1 pays in full within 30 days of accepting the data, usually as a single payment.