Notarization error: the staple and validate action failed! Error 65

[amariottini]

I'm submitting a…

• bug report
• feature request
• [X ] other

Short description of the issue/suggestion:
I didn't make a new build of my application for some months.
Now I have di error at the end of notarization process:
"The staple and validate action failed! Error 65"

Steps to reproduce the issue/enhancement:
Simply executed packageMyAppForMac task.

What is the expected behavior?
Have the app notarized

What is the current behavior?
These are the last lines of the log:

Executing command: /bin/sh -c cd '/Users/andrea/Projects/firmasmartcardagent/.' && 'xcrun' notarytool submit /Users/andrea/Projects/firmasmartcardagent/build/assets/firmasmartcardagent.app-notarization.zip --wait --keychain-profile notarytool-password
    Conducting pre-submission checks for firmasmartcardagent.app-notarization.zip and initiating connection to the Apple notary service...
    Submission ID received
      id: 7210bde9-0c6a-4e7f-ba16-178b06659599
    Successfully uploaded file
      id: 7210bde9-0c6a-4e7f-ba16-178b06659599
      path: /Users/andrea/Projects/firmasmartcardagent/build/assets/firmasmartcardagent.app-notarization.zip
    Waiting for processing to complete.
    
    Current status: In Progress...
    Current status: In Progress....
    Current status: In Progress.....
    Current status: In Progress......
    Current status: In Progress.......
    Current status: In Progress........
    Current status: In Progress.........
    Current status: In Progress..........
    Current status: In Progress...........
    Current status: In Progress............
    Current status: In Progress.............
    Current status: In Progress..............
    Current status: In Progress...............
    Current status: In Progress................
    Current status: In Progress.................
    Current status: In Progress..................
    Current status: In Progress...................
    Current status: In Progress....................
    Current status: In Progress.....................
    Current status: In Progress......................
    Current status: In Progress.......................
    Current status: In Progress........................
    Current status: In Progress.........................
    Current status: In Progress..........................
    Current status: In Progress...........................
    Current status: In Progress............................
    Current status: In Progress.............................
    Current status: In Progress..............................
    Current status: In Progress...............................
    Current status: In Progress................................
    Current status: Invalid.................................Processing complete
      id: 7210bde9-0c6a-4e7f-ba16-178b06659599
      status: Invalid
    
    Executing command: /bin/sh -c cd '/Users/andrea/Projects/firmasmartcardagent/.' && 'xcrun' stapler staple /Users/andrea/Projects/firmasmartcardagent/build/firmasmartcardagent/firmasmartcardagent.app
    Processing: /Users/andrea/Projects/firmasmartcardagent/build/firmasmartcardagent/firmasmartcardagent.app
    CloudKit query for firmasmartcardagent.app (2/435cb5f69466b2c10ba8cbca35cd16c6807fe1be) failed due to "Record not found".
    Could not find base64 encoded ticket in response for 2/435cb5f69466b2c10ba8cbca35cd16c6807fe1be
    The staple and validate action failed! Error 65.

I executed this command: xcrun notarytool log 7210bde9-0c6a-4e7f-ba16-178b06659599
with this result:

{
  "logFormatVersion": 1,
  "jobId": "7210bde9-0c6a-4e7f-ba16-178b06659599",
  "status": "Invalid",
  "statusSummary": "Archive contains critical validation errors",
  "statusCode": 4000,
  "archiveFilename": "firmasmartcardagent.app-notarization.zip",
  "uploadDate": "2024-09-23T06:42:42.057Z",
  "sha256": "ac02687091ec4be0e07bd79c6d966d768bf3538f803fa1c6f717c0cf7b225ab0",
  "ticketContents": null,
  "issues": [
    {
      "severity": "error",
      "code": null,
      "path": "firmasmartcardagent.app-notarization.zip/firmasmartcardagent.app/Contents/Resources/Java/libs/jna-5.8.0.jar/com/sun/jna/darwin-aarch64/libjnidispatch.jnilib",
      "message": "The binary is not signed with a valid Developer ID certificate.",
      "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087721",
      "architecture": "arm64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "firmasmartcardagent.app-notarization.zip/firmasmartcardagent.app/Contents/Resources/Java/libs/jna-5.8.0.jar/com/sun/jna/darwin-aarch64/libjnidispatch.jnilib",
      "message": "The signature does not include a secure timestamp.",
      "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087733",
      "architecture": "arm64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "firmasmartcardagent.app-notarization.zip/firmasmartcardagent.app/Contents/Resources/Java/libs/jna-5.8.0.jar/com/sun/jna/darwin-x86-64/libjnidispatch.jnilib",
      "message": "The binary is not signed.",
      "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087721",
      "architecture": "x86_64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "firmasmartcardagent.app-notarization.zip/firmasmartcardagent.app/Contents/Resources/Java/libs/jna-5.8.0.jar/com/sun/jna/darwin-x86-64/libjnidispatch.jnilib",
      "message": "The signature does not include a secure timestamp.",
      "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087733",
      "architecture": "x86_64"
    }
  ]
}

Do you have outputs, screenshots, demos or samples which demonstrate the problem or enhancement?

What is the motivation / use case for changing the behavior?

Please tell us about your environment:

• JavaPackager version:
• OS version: MacOS 14.6.1
• JDK version: 1.8
• Build tool:
  • Maven
  • [X ] Gradle

Other information (e.g. related issues, suggestions how to fix, links for us to have context)

[fvarrui]

Hi @amariottini!
Please, take a look into this issue comment.
Maybe it helps!

[amariottini]

Thanks @fvarrui, but I don’t understand how to follow point 2 of the proposed solution.
Is possible to exclude some file from signing?
How?

[amariottini]

Ok I think I understood:
I have a dependency from JNA, and JNA includes the file libjnidispatch.jnilib.
The notarization process finds libjnidispatch.jnilib is not signed and so fails.
I could sign libjnidispatch.jnilib before the app is build but how can I do that?
How can I sign a dependency (moreover a transitive dependency, not directly mine) managed by Gradle before javapackager put it into the app?
@fvarrui do you have some tips or some "handle" in javapackager I could hold on to?

[amariottini]

For anyone having the same problem, I solved signing libjnidispatch.jnilib and replacing the dynamic dependency on JNA with a static dependency on the modified (signed) jar.

In configurations section:
exclude group: 'net.java.dev.jna', module: 'jna'

In dependencies section:
implementation files("lib/jna-5.8.0-signed.jar")

I produced jna-5.8.0-signed.jar in this way:

jar xf jna-5.8.0.jar
codesign -f -o runtime --entitlements <path to entitlements.plist> --timestamp -s <developerId> com/sun/jna/darwin-aarch64/libjnidispatch.jnilib
codesign -f -o runtime --entitlements <path to entitlements.plist> --timestamp -s <developerId> com/sun/jna/darwin-x86-64/libjnidispatch.jnilib
rm jna-5.8.0.jar
jar cvf jna-5.8.0-signed.jar -C ./ .

However I'd like to find a way to dynamically sign during build process.
If the dependency changes to JNA 5.15.0, I have to manually re-sign it and adapt build.gradle.

[fvarrui]

So, as far as I understood, this issue is caused by .jnilib or .dylib native libraries inside .jar files?

[fvarrui]

If so, maybe JP could do a kind of deep code signing, looking for native libraries in dependencies?

[amariottini]

Exactly.
It would be a great thing if JP could do what you say.

[grill2010]

@fvarrui is there any test version or workaround available yet? I have the exact same issue, my project uses many different 3rd Party libs with bundled native .jnilib, .dylib and .so files which are located in the jar. The notarization fails as these native libs are not signed. It seems there is no built in way to handle this correctly with the javapackager plugin yet?

[fvarrui]

@fvarrui is there any test version or workaround available yet? I have the exact same issue, my project uses many different 3rd Party libs with bundled native .jnilib, .dylib and .so files which are located in the jar. The notarization fails as these native libs are not signed. It seems there is no built in way to handle this correctly with the javapackager plugin yet?

Hi @grill2010!
Sorry 😞 there's not a patch yet ... right now the only way to avoid this issue is codesigning each JAR content manually. I'll try to write a patch ASAP, but I won't be able to test it since I haven't a Mac anymore; so I'd have to test it using GitHub Actions or something similar ... a little mess!

[grill2010]

@fvarrui Well signing the jars from the runtimeClasspath is actually not that hard, I just do it that way

tasks.register("copyAndSignLibsForMac", Copy) {
    onlyIf { Os.isFamily(Os.FAMILY_MAC) }

    from configurations.runtimeClasspath
    into "libs/macossigned/Java/libs"

    duplicatesStrategy = DuplicatesStrategy.EXCLUDE

    doFirst {
        // Check if the target directory exists, delete it if so, and recreate it
        def libsTargetDir = file("libs/macossigned/Java/libs")
        if (libsTargetDir.exists()) {
            libsTargetDir.deleteDir()
            logger.lifecycle("Deleted existing directory: ${libsTargetDir}")
        }
        libsTargetDir.mkdirs()
        logger.lifecycle("Created directory structure: ${libsTargetDir}")
    }

    doLast {
        // Process each copied JAR file to sign native libraries if present
        file("libs/macossigned/Java/libs").eachFileMatch(~/.*\.jar/) { jarFile ->
            def tempDir = file("${buildDir}/temp_${jarFile.name}")
            tempDir.mkdirs()

            // Extract JAR contents to a temporary directory
            ant.unjar(src: jarFile, dest: tempDir)

            boolean containsNativeLib = false
            tempDir.eachFileRecurse { file ->
                if (file.name.endsWith(".dylib") || file.name.endsWith(".jnilib") || file.name.endsWith(".so")) {
                    containsNativeLib = true
                    exec {
                        commandLine 'codesign', '-f', '-s', 'testId', '--timestamp', file.absolutePath
                    }
                    logger.lifecycle("Signed native library: ${file}")
                } else if (file.name in ["opencv_interactive-calibration", "opencv_annotation",
                                         "opencv_version", "opencv_visualisation", "ffmpeg", "ffprobe"]) {
                    // Delete unsupported file
                    containsNativeLib = true
                    file.delete()
                    logger.lifecycle("Deleted unsupported file: ${file}")
                }
            }

            if (containsNativeLib) {
                ant.jar(destfile: jarFile, basedir: tempDir)
                logger.lifecycle("Repacked signed JAR: ${jarFile}")
            } else {
                logger.lifecycle("Kept original JAR (no native libs): ${jarFile}")
            }

            // Clean up temporary directory
            tempDir.deleteDir()
        }
    }
}

which actually extracts the jars and signs all native libs, the only problem is that there is no way of how I can replace the

PXPlay.app/Contents/Resources/Java/libs

folder with my signed jars. I tried to set the copyDependencies to false and just add the libs manually via the additionalResources like that

    additionalResources = [
            file('libs/macossigned/Java')
    ]

Unfortunately the created app doesn't launch because of some Caused by: java.lang.ClassNotFoundException. I think this is because the runnable jar is created differently when setting the copyDependencies to false. There is also no hook or task I can bind my gradle script to so that I can execute a task as soon as the app was created but before the signing and dmg file creation starts. Any other ideas of how to make it work?

[grill2010]

Okay I actually got it working, the issue was the Manifest file which did not include the class path when setting the copyDependencies to false. I do it like that in my configuration

copyDependencies = !Os.isFamily(Os.FAMILY_MAC)

so in order to workaround this issue I use this

if (Os.isFamily(Os.FAMILY_MAC)) {
        jrePath = file('macos/jre/jre.jre') // I use a custom jre (explanation below)
        additionalResources = [
                file('macos/macossigned/Java') // this directory contains my signed jars
        ]

        def classPathJars = fileTree("macos/macossigned/Java/libs").matching {
            include '*.jar'
        }.files.collect { "libs/${it.name}" }.join(' ')

        // add the Class-Path entries manually as it will not be included when you set copyDependencies to false
        manifest {
            additionalEntries = [
                    'Class-Path': classPathJars
            ]
        }
    }

I also set a manual jre because when you set the copyDependencies to false the standard bundled jre will miss some stuff. So I built the app once with copyDependencies=true extracted the working jre folder from within the app file and copied it to my macos/jre/jre.jre folder in my project.

And as mentioned before I sign all my dependencies automatically via this task

tasks.register("copyAndSignLibsForMac", Copy) {
    onlyIf { Os.isFamily(Os.FAMILY_MAC) }

    from configurations.runtimeClasspath
    into "macos/macossigned/Java/libs"

    duplicatesStrategy = DuplicatesStrategy.EXCLUDE

    doFirst {
        // Check if the target directory exists, delete it if so, and recreate it
        def libsTargetDir = file("macos/macossigned/Java/libs")
        if (libsTargetDir.exists()) {
            libsTargetDir.deleteDir()
            logger.lifecycle("Deleted existing directory: ${libsTargetDir}")
        }
        libsTargetDir.mkdirs()
        logger.lifecycle("Created directory structure: ${libsTargetDir}")
    }

    doLast {
        // Process each copied JAR file to sign native libraries if present
        file("macos/macossigned/Java/libs").eachFileMatch(~/.*\.jar/) { jarFile ->
            def tempDir = file("${buildDir}/temp_${jarFile.name}")
            tempDir.mkdirs()

            // Extract JAR contents to a temporary directory
            ant.unjar(src: jarFile, dest: tempDir)

            boolean containsNativeLib = false
            tempDir.eachFileRecurse { file ->
                if (file.name.endsWith(".dylib") || file.name.endsWith(".jnilib") || file.name.endsWith(".so")) {
                    containsNativeLib = true
                    exec {
                        commandLine 'codesign', '-f', '-s', 'dummyId', '--timestamp', file.absolutePath
                    }
                    logger.lifecycle("Signed native library: ${file}")
                } else if (file.name in ["opencv_interactive-calibration", "opencv_annotation",
                                         "opencv_version", "opencv_visualisation", "ffmpeg", "ffprobe"]) {
                    if (!file.absolutePath.contains("META-INF") && file.isFile()) {
                        containsNativeLib = true
                        file.delete()
                        logger.lifecycle("Deleted unsupported file: ${file}")
                    } else {
                        logger.lifecycle("Skipped file in META-INF or directory: ${file}")
                    }
                }
            }

            // If the JAR contains signed native libs, repack it; otherwise, keep the original JAR
            if (containsNativeLib) {
                ant.jar(destfile: jarFile, basedir: tempDir)
                logger.lifecycle("Repacked signed JAR: ${jarFile}")
            } else {
                logger.lifecycle("Kept original JAR (no native libs): ${jarFile}")
            }

            // Clean up temporary directory
            tempDir.deleteDir()
        }
    }
}

// Run copyAndSignLibsForMac before build and all its dependencies
tasks.named("build") {
    dependsOn("copyAndSignLibsForMac")
}

I also updated the section for deleting unsupported files after realizing it was unintentionally removing contents from the META-INF directory and entire directories. This update is specific to my project, but I hope it’s useful information.

If you consider adding similar functionality to javapackager, it would be incredibly helpful to have an option for excluding certain files from JARs. In my case, some third-party JARs contained files that couldn't be signed and were subsequently rejected during notarization.

[fvarrui]

If you consider adding similar functionality to javapackager, it would be incredibly helpful to have an option for excluding certain files from JARs. In my case, some third-party JARs contained files that couldn't be signed and were subsequently rejected during notarization.

Hi @grill2010!
Wow! Great job ... Ok, I'll keep in mind your suggestion.
Thanks!!!