I am the maintainer of a public GitHub repo. I have set up GitHub Actions to build a publish to GitHub Packages. You can see the package has been created here:
https://github.com/paulschwarz/spring-dotenv/packages/135114
The first thing I notice is that GitHub only gives a Maven installation snippet. I used this code to add the dependency to another project and it appeared to work.
Now I want to import this package into a Gradle project. I added
dependencies {
implementation ('me.paulschwarz:spring-dotenv:0.0.3')
}
and gradle tells me
Could not find me.paulschwarz:spring-dotenv:0.0.3.
Searched in the following locations:
- https://jcenter.bintray.com/me/paulschwarz/spring-dotenv/0.0.3/spring-dotenv-0.0.3.pom
- https://repo.maven.apache.org/maven2/me/paulschwarz/spring-dotenv/0.0.3/spring-dotenv-0.0.3.pom
This is already strange because my Maven project appeared to have no problem resolving the dependency. I must say I'm curious how that worked? Surely GitHub Packages isn't integrated with JCenter or Maven Central?
Anyway, next step, add the repository
repositories {
jcenter()
mavenCentral()
maven { url = uri('https://maven.pkg.github.com/paulschwarz/spring-dotenv') }
}
At this point, Gradle should understand where to find the package. However, I get this
> Could not resolve me.paulschwarz:spring-dotenv:0.0.3.
> Could not get resource 'https://maven.pkg.github.com/paulschwarz/spring-dotenv/me/paulschwarz/spring-dotenv/0.0.3/spring-dotenv-0.0.3.pom'.
> Could not GET 'https://maven.pkg.github.com/paulschwarz/spring-dotenv/me/paulschwarz/spring-dotenv/0.0.3/spring-dotenv-0.0.3.pom'. Received status code 401 from server: Unauthorized
Is this really a 401 unauthorized? or is the URL wrong and it's trying to hit an authorized endpoint?
If it's genuine, then why? This is a public repo with public packages. I can download the package directly from the GitHub page anonymously. What am I doing wrong in Gradle?
As you have observed, GitHub doesn't support unauthorized package access right now (but planned in future) as explained by one of their staff (May 27 '20):
Our Maven service doesn’t allow for unauthorized access right now. We plan to offer this in the future but need to improve the service a bit before that.
For Actions you can add a PAT to your secrets store or use the GITHUB_TOKEN to authenticate. In your settings.xml we suggest using the environment variable approach (see setup-java 4) so you don’t store the tokens in the file.
As mentioned above you need to authenticate to GitHub Packages.
ext {
GITHUB_TOKEN = System.getenv("GITHUB_TOKEN")
}
maven {
url "https://maven.pkg.github.com/paulschwarz/spring-dotenv"
credentials {
username GITHUB_USER
password GITHUB_TOKEN
}
}
Where GITHUB_USER is defined in your gradle.properties and GITHUB_TOKEN is defined as an environment variable. GITHUB_TOKEN is available inside your GitHub Actions workflow file as ${{ secrets.GITHUB_TOKEN }}
You will have to define it yourself when running locally.
In my case, I'm using Maven. After researching around, ended up that I need to generate a GitHub token instead of using the plain GitHub user login password one.
I do publish an open source library to Maven Central. In order to do that the gradle.build file contains variable references to a gradle.properties file which contains secret information like usernames and passwords.
Of course the build.gradle needs to be published to the public git repository, the gradle.properties should not be published, due to containing all the personal information but without the properties file the build.gradle is not valid.
How are open source projects handling those sensitive data?
Here is a solution based on this answer, with the use of findProperty method to allow users to build your project without providing the publishing credentials (issue you mentioned in your comment above)
move credentials outside the project's gradle.properties and put them to your local user /.gradle/gradle.properties configuration file
in your publish task definition, use:
authentication(userName: findProperty('mavenUser'), password: findProperty('mavenPassword'))
For my personal project, I am using Circle.ci to test and deploy the OSX application (with upload hook to S3, to distribute it outside the AppStore - using the signed with Developer ID distribution method). I am using FastLane to build the app, which works flawlessly when building it from my local terminal, but I cannot get past code signing step when using the CI server.
The issue is very simple, the keychain does not import the provision profile, which I can tell from the pre-build step:
1 key imported.
No provisioning profiles found in repository.
You must add a provisioning profile to your repository
to enable CircleCI code-signing support.
Currently installed Code-Signing identities:
Policy: Code Signing
Matching identities
0 identities found
Valid identities only
0 valid identities found
I have my .p12 with private key and certificate uploaded to the repository, no issue there. I have also added the .provisionprofile to the repository (root, even different locations) but to no avail. The documentation states (https://circleci.com/docs/ios-code-signing/) that there is a need for .mobileprovision, but that only covers the iOS, not the OSX, while the system should be nearly identical so I assumed that would work as well.
So the TLDR question is: How do I import the provisioning profile of the OSX distribution on the Circle.CI, so the keychain accepts the entry? Or is there a way around it - like skipping the signing step (disabling it in XCode) and signing it by hand with some .sh?
Thanks everyone!
I am sharing my experience of deploying iOS app to Testflight using CircleCI. Maybe this helps.
Put the provision profile file in the root of your project directory. Make sure it is a normal "Distribution Profile" not a App Store submission profile.
Just add the .p12 key in the project settings of CircleCI. Remove all other certificates / .p12 file from the repository.
I tried using fastlane to build and distribute the build but it was failing in CircleCI. I had these three steps in my Fastfile beta environment.
increment_build_number
gym(scheme: "myScheme”)
pilot
I changed that to just
increment_build_number
My Circle CI file
machine:
environment:
GYM_CODE_SIGNING_IDENTITY: "iPhone Distribution: Company Name (XXXXXX)"
xcode:
version: "8.0"
test:
override:
- set -o pipefail && xcodebuild -workspace 'my.xcworkspace' -scheme 'myscheme' clean build test -sdk iphonesimulator -destination 'id=AC291080-8EFE-4095-8C55-B1E952EFFC36' CODE_SIGNING_REQUIRED=NO CODE_SIGN_IDENTITY= PROVISIONING_PROFILE=
dependencies:
pre:
- gem update fastlane
deployment:
beta_distribution:
branch: master
commands:
- fastlane beta
- gym --scheme "myscheme" --workspace "my.xcworkspace" --export_method app-store --use_legacy_build_api false
- pilot upload
With the release of Xcode 8, Apple introduced a new way of managing the signing configuration. Now you have two options Manual and Automatic.
According to the WWDC 2016 Session about Code signing (WWDC 2016 - 401 - What's new in Xcode app signing), when you select Automatic signing, Xcode is going to:
Create signing certificates
Create and update App IDs
Create and update provisioning profiles
But according to what Apple says in that session, the Automatic Signing is going to use Development signing and will be limited to Xcode-created provisioning profiles.
The issue comes when you try to use Automatic Signing on a CI environment (like Travis CI or Jenkins). I'm not able to figure out an easy way to keep using Automatic and sign for Distribution (as Xcode forces you to use Development and Xcode-created provisioning profiles).
The new "Xcode-created provisioning profiles" do not show up in the developer portal, although I can find then in my machine... should I move those profiles to the CI machine, build for Development and export for Distribution? Is there a way to override the Automatic Signing using xcodebuild?
I basically run into the same issue using Jenkins CI and the Xcode Plugin.
I ended up doing the build and codesigning stuff myself using xcodebuild.
0. Prerequisites
In order to get the following steps done successfully, you need to have installed the necessary provisioning profiles and certificates. That means your code signing should already be working in general.
1. Building an .xcarchive
xcodebuild -project <path/to/project.xcproj> -scheme <scheme-name> -configuration <config-name> clean archive -archivePath <output-path> DEVELOPMENT_TEAM=<dev-team-id>
DEVELOPMENT_TEAM: your 10 digit developer team id (something like A1B2C3D4E5)
2. Exporting to .ipa
xcodebuild -exportArchive -archivePath <path/to/your.xcarchive> -exportOptionsPlist <path/to/exportOptions.plist> -exportPath <output-path>
Example of an exportOptions.plist:
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>method</key>
<string>development</string>
<key>teamID</key>
<string> A1B2C3D4E5 </string>
</dict>
</plist>
method: is one of development, app-store, ad-hoc, enterprise
teamID: your 10 digit developer team id (something like A1B2C3D4E5)
This process is anyway closer to what you would do with Xcode manually, than what for example the Jenkins Xcode Plugin does.
Note: The .xcarchive file will always be develpment signed, but selecting "app-store" as method in the 2nd step will do the correct distribution signing and also include the distribution profile as "embedded.mobileprovision".
Hope this helps.
After trying a few options, these are the solutions that I was able to use on my CI server:
Include the Developer certificate and private key as well as the auto generated provisioning profiles in the CI environment:
Using Automatic signing forces you to use a Developer certificate and auto-generated provisioning profiles. One option is to export your development certificate and private key (Application -> Utilities -> Keychain Access) and the auto-generated provisioning profiles to the CI machine. A way to locate the auto-generated provisioning profiles is to navigate to ~/Library/MobileDevice/Provisioning\ Profiles/, move all files to a backup folder, open Xcode and archive the project. Xcode will create auto-generated development provisioning profiles and will copy them to the Provisioning Profiles folder.
xcodebuild archive ... will create a .xcarchive signed for Development. xcodebuild -exportArchive ... can then resign the build for Distribution
Replace 'Automatic' with 'Manual' when building on a CI environment
Before calling xcodebuild a workaround is to replace all instances of ProvisioningStyle = Automatic with ProvisioningStyle = Manual in the project file. sed can be used for a simple find an replace in the pbxproj file:
sed -i '' 's/ProvisioningStyle = Automatic;/ProvisioningStyle = Manual;/' <ProjectName>.xcodeproj/project.pbxproj
#thelvis also created a Ruby script to do this using the xcodeproj gem. The script gives you a better control over what is changed.
xcodebuild will then use the code signing identity (CODE_SIGN_IDENTITY) set in the project, as well as the provisioning profiles (PROVISIONING_PROFILE_SPECIFIER). Those settings can also be provided as parameters to xcodebuild and they will override the code signing identity and/or provisioning profile set in the project.
EDIT: with Xcode 9, xcodebuild has a new build settings parameter CODE_SIGN_STYLE to select between Automatic and Manual so there's no need to find and replace instances of automatic with manual in the project file, more info in WWDC 2017 Session 403 What's New in Signing for Xcode and Xcode Server
Switch to manual signing
Manual signing will provide total control over the code signing identities and provisioning profiles being used. It's probably the cleanest solution, but with the downside of losing all the benefits of Automatic signing.
To learn more about code signing with Xcode 8 I really recommend this article as well as the WWDC2016 session 401 - What's new in Xcode app signing
I'm considering another option I've not seen mentioned here yet. Setup two identical targets, that only differ in their signing settings.
Development Target uses automatic signing to get all of those benefits when new devices / developers are added
CI Target uses manual signing
Downside is that you would have to manage two identical targets. Upside is that get the benefits of automatic signing for development, and don't have to maintain potentially brittle scripts that modify your project just before build time.
If you are using Xcode 8.x and Jenkins for CI. Then probably you would face issue with "Signing for “YourProjectName" requires a development team. Select a development team in the project editor.
Code signing is required for product type 'Application' in SDK 'iOS 10.1’”.** BUILD FAILED ** when running the job.
What is the solution?.
Solution is:
set Provisioning profile to None in Xcode project build settings.
In jenkins, Create a execute shell before the Xcode setting and write the below command
sed -i '' 's/ProvisioningStyle = Automatic;/ProvisioningStyle = Manual;/' ProjectName.xcodeproj/project.pbxproj
Remember: keep that execute shell before Xcode settings in Build section of jenkins.
This works.
There is one more way to workaround nonfunctional signing in CI/CD pipeline when the signing specified in the target (or project) fails.
We use automatic signing for our apps to have a good developer experience, and in CI/CD only signing is done in the last step when .ipa is produced.
Produce an unsigned .xcarchive
xcodebuild -workspace Runner.workspace CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO CODE_SIGNING_ALLOWED=NO <more parameters follow>
Sign the final .ipa file with configuration in exportOptions.plist
xcodebuild -exportArchive -archivePath <.xcarchive> -exportOptionsPlist <exportOptions.plist> -exportPath <output-path>
For me, nothing worked. I solved my problem by changing a file in Xcode app installed on your Mac Mini (CI server with Jenkins), as shown in this link:
https://www.jayway.com/2015/05/21/fixing-your-ios-build-scripts/
Additionally I turned off automatic signing from Xcode.
All done! Finally works!
I noticed my Unity build was never adding a ProvisioningStyle key to my XCode project. I then found a way to manually add the ProvisioningStyle by using a "PostProcessBuild" build script. i.e. a unit of code that is called after the IOS XCode project has been built by Unity.
First I had a look at what the project.pbxproj file should look like - when it is set to Manual Provisioning:
/* Begin PBXDictionary section */
29B97313FDCFA39411CA2CEA /* Project object */ = {
isa = PBXProject;
attributes = {
TargetAttributes = {
1D6058900D05DD3D006BFB54 /* Unity-iPhone */ = {
ProvisioningStyle = Manual;
};
5623C57217FDCB0800090B9E /* Unity-iPhone Tests */ = {
TestTargetID = 1D6058900D05DD3D006BFB54 /* Unity-iPhone */;
};
};
};
Then I created my code to replicate the "structure" of the file seen above. (using the XCodeEditor project found here: XCodeEditor)
[PostProcessBuild]
public static void OnPostProcessBuild(BuildTarget target, string path)
{
// Create a new project object from build target
XCProject project = new XCProject(path);
if (target == BuildTarget.iOS)
{
//Add Manual ProvisioningStyle - this is to force manual signing of the XCode project
bool provisioningSuccess = AddProvisioningStyle(project, "Manual");
if (provisioningSuccess)
project.Save();
}
}
private static bool AddProvisioningStyle(XCProject project, string style)
{
var pbxProject = project.project;
var attr = pbxProject.data["attributes"] as PBXDictionary;
var targetAttributes = attr["TargetAttributes"] as PBXDictionary;
var testTargetIDGuid = FindValue(targetAttributes, "TestTargetID");
if (!string.IsNullOrEmpty(testTargetIDGuid))
{
var settings = new PBXDictionary();
//here we set the ProvisioningStyle value
settings.Add("ProvisioningStyle", style);
targetAttributes.Add(testTargetIDGuid, settings);
var masterTest = FindValue(targetAttributes, "ProvisioningStyle");
if (masterTest == style)
{
return true;
}
}
return false;
}
private static string FindValue(PBXDictionary targetAttributes, string key)
{
foreach (var item in targetAttributes)
{
var ma = item.Value as PBXDictionary;
foreach (var di in ma)
{
var lookKey = di.Key;
if (lookKey == key)
{
return di.Value.ToString();
}
}
}
return "";
}
What fixed it for me was this: http://code-dojo.blogspot.jp/2012/09/fix-ios-code-signing-issue-when-using.html
... copying certificates from Login keychain to System keychain.
You might also want to set all dev certificates to 'Allow all applications to access this item' (Right-click/Get Info/Access Control).
There is a tool called fastlane which makes using xcodebuild much easier and it is maintained meaning new updates will continue to provide support for changes to xcode. It makes it much easier to create scripts and config for building and codesigning your app among many other xcode automation tools it supports. I'd recommend giving it a look into.
I would like to use credentials of the physical user during the release. I use following stack: my own maven plugin wrapping maven-release-plugin, jenkins, nexus, git. On Jenkins I use parameters and password mask plugin to provide username and password for the user who is executing the build. The problem is that credentials for Jenkins, Git and Nexus may be different, so I can NOT re-use them. Obviously I could use CI user to authenticate everything but because of some regulations I can not do that.
First I tried to use -Dusername and -Dpassword but 1) this works only for git, 2) password for git and nexus is different. Besides that I want to use key based authentication for git.
Then I tried to programmatically change credentials by changing session/project object before executing maven-release-plugin:
final MavenExecutionRequest request = new DefaultMavenExecutionRequest()
.setLocalRepository(session.getLocalRepository())
.setPluginArtifactRepositories(session.getCurrentProject().getPluginArtifactRepositories())
.setRemoteRepositories(session.getCurrentProject().getRemoteArtifactRepositories())
.setBaseDirectory(new File(properties.getCheckoutDirectory()))
.setPluginGroups(session.getPluginGroups())
.setProjectPresent(true)
.setPom(new File(properties.getCheckoutDirectory() + "/pom.xml"))
.setGoals(Arrays.asList(goal))
.setInteractiveMode(false)
.setUserSettingsFile(new File(session.getSystemProperties().getProperty("user.home") + "/.m2/settings.xml"))
;
request.setUserProperties(preparePropertiesForMvn(properties));
for (ArtifactRepository artifactRepository : session.getCurrentProject().getRemoteArtifactRepositories()) {
if (artifactRepository.getId().equals("nexus-releases")) {
artifactRepository.setAuthentication(new Authentication(properties.getNexusUser(),properties.getNexusPassword()));
}
}
final MavenExecutionResult result = defaultMaven.execute(request);
At very end stage maven-deploy-plugin (which is executed by release plugin) reads ~/.m2/settings.xml file and gets (wrong - CI) credentials from it.
Is there the other way than preparing own customized settings.xml file for the build with builded in credentials? I would like to avoid to store the credentials on the local storage.
You can use the following method to add your own server configuration:
org.apache.maven.settings.Settings.addServer(org.apache.maven.settings.Server)
Tested and verified with help:effective-settings as I have no means to test agains any repository manager at the moment.
As long as you call the above method with your own server configuration ahead of the Maven deploy plugin, you should be good to go.
Server server = new Server();
server.setId("myServer");
server.setUsername("myUserName");
server.setPassword("myPassword");
settings.addServer(server);