Different errors and inconsistency while trying to push files to remote using GitHub token.

[praveenkanna]

I'm a newbie to pygit2 and seeing different errors while pushing to remote, some times it succeeds and sometimes it is raising errors. I'm using the python Python 3.7.7 in OSX, pygit2==1.2.0, pip version: pip 20.0.2. Here are the below errors:

Pushing updated files to remote:
   - Added: git_update.py to the repository
   - Commited: git_update.py to the repository
   - Pushing files to the remote repository
Traceback (most recent call last):
  File "git_update.py", line 98, in <module>
    pull_push_new_commits()
  File "git_update.py", line 94, in pull_push_new_commits
    push_new_commits(repo, remote, callbacks)
  File "git_update.py", line 77, in push_new_commits
    remote.push([repo.head.name], callbacks=callbacks)
  File "/usr/local/lib/python3.7/site-packages/pygit2/remote.py", line 536, in push
    check_error(err)
  File "/usr/local/lib/python3.7/site-packages/pygit2/errors.py", line 61, in check_error
    raise GitError(message)
_pygit2.GitError: unexpected EOF

Pushing updated files to remote:
   - Added: git_update.py to the repository
   - Committed: git_update.py to the repository
   - Pushing files to the remote repository
Fatal Python error: ffi.from_handle() detected that the address passed points to garbage. If it is really the result of ffi.new_handle(), then the Python object has already been garbage collected

Current thread 0x000000011350fdc0 (most recent call first):
  File "/usr/local/lib/python3.7/site-packages/pygit2/remote.py", line 317 in _certificate_cb
  File "/usr/local/lib/python3.7/site-packages/pygit2/remote.py", line 535 in push
  File "git_update.py", line 77 in push_new_commits
  File "git_update.py", line 94 in pull_push_new_commits
  File "git_update.py", line 98 in <module>
Abort trap: 6

Below is the code:

import pygit2
import datetime
import os


def git_url_update(repo):
    repo_url = repo.remotes[0].url
    repo_url_list = repo_url.split('@')

    if repo_url.startswith('git'):
        repo_url_path = repo_url_list[0].replace('git', 'https://')
        repo_path_change = repo_url_list[1].replace(':', '/')
        remote_url = repo_url_path + repo_path_change

    elif len(repo_url_list) == 1:
        remote_url = repo_url_list[0]
    else:
        if ':' in repo_url_list[1]:
            repo_path_change = repo_url_list[1].replace(':', '/')
        else:
            repo_path_change = repo_url_list[1]
        remote_url = repo_url_list[0] + '@' + \
            repo_path_change

    repo.remotes.set_url('origin', remote_url)
    repo.remotes.set_push_url('origin', remote_url)


def update_local_repo(repo, remote, callbacks):
    remote.fetch(callbacks=callbacks)
    remote_master_id = repo.lookup_reference(
        'refs/remotes/origin/%s' % (repo.head.shorthand)).target
    merge_result, _ = repo.merge_analysis(remote_master_id)

    if merge_result & pygit2.GIT_MERGE_ANALYSIS_UP_TO_DATE:
        print("current directory is up to date")
    elif merge_result & pygit2.GIT_MERGE_ANALYSIS_FASTFORWARD:
        repo.checkout_tree(repo.get(remote_master_id))
        try:
            master_ref = repo.lookup_reference(
                'refs/heads/%s' % (repo.head.shorthand))
            master_ref.set_target(remote_master_id)
        except KeyError:
            repo.create_branch(repo.head.shorthand, repo.get(remote_master_id))
        repo.head.set_target(remote_master_id)
    else:
        raise AssertionError('Unknown merge analysis result')


def push_new_commits(repo, remote, callbacks):
    print('\nPushing updated files to remote:')
    date_object = datetime.date.today()
    commit_message = 'Snapshot {date_object}'.format(
        date_object=date_object)

    index = repo.index
    status = repo.status()

    for changed_file in status:
        if not repo.path_is_ignored(changed_file):
            index.add(changed_file)
            index.write()
            print('   - Added: {} to the repository'.format(changed_file))

    tree = index.write_tree()
    author = pygit2.Signature('<user>', '<email>')
    commiter = pygit2.Signature('<user>', '<email>')

    print('   - Commited: {} to the repository'.format(changed_file))
    oid = repo.create_commit(repo.head.name,
                             author, commiter, commit_message, tree, [repo.head.target])

    repo.head.set_target(oid)
    print('   - Pushing files to the remote repository')
    remote.push([repo.head.name], callbacks=callbacks)


def pull_push_new_commits():
    currentDirectory = os.getcwd()
    print("Pulling the repo for new changes")
    repository_path = pygit2.discover_repository(currentDirectory)
    repo = pygit2.Repository(repository_path)
    git_url_update(repo)
    remoteName = repo.remotes[0].name
    remote = repo.remotes[remoteName]

    callbacks = pygit2.RemoteCallbacks(pygit2.UserPass(
        '<TOKEN>', 'x-oauth-basic'))

    update_local_repo(repo, remote, callbacks)
    push_new_commits(repo, remote, callbacks)


if __name__ == "__main__":
    pull_push_new_commits()

I'm tweaking the git URL because of using only Github token since we want to run this as a cron job inside the docker container and our pipeline uses a different user and I can't copy that ssh keys of that user those to the container. (lot of reasons) so I chose GitHub token approach since it is the easiest approach for me.

Also, I'm seeing SSL unknown error inside the python docker container ( tried with different flavours )

Pushing updated files to remote:
   - Added: Dockerfile to the repository
   - Commited: Dockerfile to the repository
   - Pushing files to the remote repository
Traceback (most recent call last):
  File "git_update.py", line 98, in <module>
    pull_push_new_commits()
  File "git_update.py", line 94, in pull_push_new_commits
    push_new_commits(repo, remote, callbacks)
  File "git_update.py", line 77, in push_new_commits
    remote.push([repo.head.name], callbacks=callbacks)
  File "/usr/local/lib/python3.7/site-packages/pygit2/remote.py", line 536, in push
    check_error(err)
  File "/usr/local/lib/python3.7/site-packages/pygit2/errors.py", line 61, in check_error
    raise GitError(message)
_pygit2.GitError: SSL error: unknown error

To explain a bit more about this code, I'm pulling the new changes from remote and run a script which updates some files or creates new files, commits and pushes back to the repository.

Tried different things but seeing different issues and been blocked from a week so posting for help. Can anyone help me out, if I'm missing anything or doing any mistake?

[praveenkanna]

Hi @jdavid , Can you please help me on this, this was blocked for more than a week. All the installation looks good and it gives me weird errors only while pushing to remote all other things are good and I don't understand where I'm doing wrong. Tried other python modules for git but this looks promising to me so reaching out to you.

[jdavid]

@praveenkanna can you try with the master branch? There have been many changes related to callbacks.

If it doesn't work with master either, then try with version 1.1.1, to see whether it's a regression. Version 1.2.0 is the first one to include wheels for Mac OS, it may be related to the problems you find.

[praveenkanna]

@jdavid Thanks for your response. I tried using the master's changes and also 1.1.1, i'm still getting this error with both the versions:

Fatal Python error: ffi.from_handle() detected that the address passed points to garbage. If it is really the result of ffi.new_handle(), then the Python object has already been garbage collected

Current thread 0x000000011350fdc0 (most recent call first):
  File "/usr/local/lib/python3.7/site-packages/pygit2/remote.py", line 317 in _certificate_cb
  File "/usr/local/lib/python3.7/site-packages/pygit2/remote.py", line 535 in push
  File "git_update.py", line 77 in push_new_commits
  File "git_update.py", line 94 in pull_push_new_commits
  File "git_update.py", line 98 in <module>
Abort trap: 6

Also, inside container everything worked properly when changing from LIBGIT_VERSION 1.0.0 to LIBGIT_VERSION 0.99.0 with pygit2==1.2.0.

[praveenkanna]

@jdavid Don't know what's happening here, not sure what's wrong here. Because if don't try to update the local repo everything working fine inside the container and in my mac. If i try to update the local repository i'm seeing SSL unknown error and Python error: ffi.from_handle() errors.

Am i doing anything wrong ?

[jdavid]

Tested your code here, with Linux, but it worked. Ping @rcoup in case he has an idea

You can (with the master branch):

• Verify you are using the latest version of cffi
• Try to reproduce with Linux, since that's what I have
• Provide the shortest program that triggers the error (the one you've post is pretty long)
• Post again the error you get, with the master branch, so line numbers are the same

If I cannot reproduce the error you'll have to debug it:

• The unexpected EOF error comes from libgit2
• The from_handle error comes from libgit2, you'll see that there's a single place in the master branch where this is called.

[rcoup]

👋

@praveenkanna if you can simplify your program (eg: your last message suggests pull-then-push triggers it?) then that would help immensely.

The container is Linux and your machine is a Mac, so whatever the issue is looks like it isn't OS-specific.

i'm seeing SSL unknown error and Python error: ffi.from_handle() errors

How do you see both errors? Do you get different results sometimes, or is one on macOS and another Linux in the container?

[praveenkanna]

@rcoup @jdavid

I tried with current version of libgit2 and master branch of pygit2 in mac and python:3.8.2-slim-buster docker image. Below are the errors i see:

I'm seeing either 'SSL unknown error' or 'Python error: ffi.from_handle()'.
mostly i see SSL unknown error when i run the script in container.

Actual code workflow looks like below:

1. update the url and get the required format.
2. update the local repo by fetching the new changes from remote.
3. run our python method which updates the local files if any.
4. check for the updates in the local repository and get the files needed to be committed.
5. add, commit the files which are not ignored and push to the remote repository.

One thing i have observed is:
If i run the script with removing the second step that is updating the local repository, everything works as expected. I don't see any errors. But when i update the local repo and try to commit and push the files i see below error.

Traceback (most recent call last):
  File "git_update.py", line 107, in <module>
    pull_push_new_commits()
  File "git_update.py", line 103, in pull_push_new_commits
    push_new_commits(repo, remote, callbacks,credentials)
  File "git_update.py", line 86, in push_new_commits
    remote.push([ref], callbacks=callbacks)
  File "/usr/local/lib/python3.8/site-packages/pygit2/remote.py", line 224, in push
    payload.check_error(err)
  File "/usr/local/lib/python3.8/site-packages/pygit2/callbacks.py", line 93, in check_error
    check_error(error_code)
  File "/usr/local/lib/python3.8/site-packages/pygit2/errors.py", line 65, in check_error
    raise GitError(message)
_pygit2.GitError: SSL error: unknown error

[praveenkanna]

Hi @rcoup @jdavid I have observed two things. when i try the same code in external github that is external to corporate proxy i don't see any errors. I tried adding proxy settings to git config and i don't see SSL anymore.

But still i see cffi error like below:

Fatal Python error: ffi.from_handle() detected that the address passed points to garbage. If it is really the result of ffi.new_handle(), then the Python object has already been garbage collected
Python runtime state: initialized

Current thread 0x00007f4bd0b80740 (most recent call first):
  File "/usr/local/lib/python3.8/site-packages/pygit2/callbacks.py", line 328 in wrapper
  File "/usr/local/lib/python3.8/site-packages/pygit2/remote.py", line 223 in push
  File "git_update.py", line 116 in <module>
Aborted

Any suggestions in getting rid of this issue please.

[jdavid]

Since you can reproduce it, it should be relatively easy to fix. But first you need to understand how new_handle and from_handle work, please read https://cffi.readthedocs.io/en/latest/ref.html#ffi-new-handle-ffi-from-handle

Then, from your traceback it starts in pygit2/remote.py:

    def push(self, specs, callbacks=None):
        with git_push_options(callbacks) as payload:
            opts = payload.push_options
            with StrArray(specs) as refspecs:
                err = C.git_remote_push(self._remote, refspecs, opts)
                payload.check_error(err)

The handle is created in the call to git_push_options:

@contextmanager
def git_push_options(payload, opts=None):
    [...]
    opts = ffi.new('git_push_options *')
    [...]
    # Plug callbacks
    opts.callbacks.sideband_progress = C._sideband_progress_cb
    opts.callbacks.transfer_progress = C._transfer_progress_cb
    opts.callbacks.update_tips = C._update_tips_cb
    opts.callbacks.credentials = C._credentials_cb
    opts.callbacks.certificate_check = C._certificate_cb
    opts.callbacks.push_update_reference = C._push_update_reference_cb
    # Payload
    handle = ffi.new_handle(payload)
    opts.callbacks.payload = handle
    [...]
    payload.push_options = opts
    [...]
    yield payload

So the handle is supposed to survive until the context manager exits: it will yield the payload, then C.git_remote_push will be called, then the context manager will exit and the handle will be garbage collected.

But the error says that the handle has already been garbage collected when C.git_remote_push is called: C.git_remote_push is the libgit2 function, it will call a number of pygit2 callbacks, see the Plug callbacks section above.

One thing you can try is to identify which one of these callbacks is failing, because that information doesn't appear in the traceback. Let's take an example:

@libgit2_callback
def _sideband_progress_cb(string, length, data):
    progress = getattr(data, 'progress', None)
    if not progress:
        return 0
    
    s = ffi.string(string, length).decode('utf-8')
    progress(s)
    return 0

You see these callbacks are all wrapped within the libgit2_callback decorator:

def libgit2_callback(f):
    @wraps(f)
    def wrapper(*args):
        data = ffi.from_handle(args[-1])
        args = args[:-1] + (data,)
        try:
            return f(*args)
        except Passthrough:
        [...]

    return ffi.def_extern()(wrapper)

That's where the call to ffi.from_handleappears, and the error happens, args[-1] has already been garbage collected but it should not.

Summarising this is the flow:

> Remote.push(...)
    > git_push_options(...)  # the handle is created here
        > C.git_remote_push(...) # this one is from libgit2
            > libgit2_callback(...) # the handle is retrieved here
                > pygit2 callbacks, e.g. _sideband_progress_cb(...) # these may call user defined functions
    < git_push_options(...)  # the handle is garbage collected here

You had the same error with 1.2.0, so it's not a regression even though this code was completely rewritten in 1.2.1; the old code has been basically the same since many years ago, and it's the first time this problem has been reported, that I remember.

Since this is strongly related to cffi, please verify you're using the latest version of cffi.

[jdavid]

Hi @rcoup @jdavid I have observed two things. when i try the same code in external github that is external to corporate proxy i don't see any errors. I tried adding proxy settings to git config and i don't see SSL anymore.

Actually pygit2 didn't support proxy until today, feature added in PR #1063
Closing, reopen if it still doesn't work with the latest commit.

[slackner]

I've also gotten errors like:

Fatal Python error: b_from_handle: ffi.from_handle() detected that the address passed points to garbage. If it is really the result of ffi.new_handle(), then the Python object has already been garbage collected

I've tracked it down to a libgit2 bug, and opened a merge request for it: libgit2/libgit2#6416

@praveenkanna If you're still interested in this issue, feel free to test my proposed merge request.

[praveenkanna]

Ah! not interested on this now, I have used other module for this work earlier. Anyways thanks for the update.