Using Springframework Repo central in China is a pain. I am thinking about to build a mirror for springfw repo.
Like there are plenty of mirrors around the around for maven center repo.
And I know the structure for maven center and springfw repo are both artifactory from Jfog.
So I think it is possible to set up a mirror for springfw, right?
I've used Sonatype Nexus to do this before, it's available in an open source version that you can run yourself.
I'd recommend setting it up in proxy mode which means that you will fetch all your artifacts from it and it will in turn fetch artifacts from the upstream mirror the first time and then serve them from a local cache in the future.
Related
I'm trying to understand how to work with a remote repository in Artifactory for a Go project. My initial expectation was that it'll work transparently, all I would need to do is to point GORPOXY variable to a virtual repository (with local and remote behind it), do go build and dependencies will either be downloaded from the Artifactory cache or Artifactory would download them transparently. Similar to the way it works for maven dependencies.
When I tried that, it complained that the dependencies weren't found in artifactory. Ok.
Reading the documentation two things stand out. First, there's nothing there about GOPROXY and everything is about using artifactory cli. That's a big downside for several reasons.
Second, is that you need to publish dependencies manually with jfrog rt go-publish go --self=false --deps=ALL and then dependencies appear under a local repository.
So I'm trying to figure out if 1) I can avoid using JFrog CLI and 2) what's the point of remote repositories if they don't proxy? Or maybe I'm missing something?
Artifactory 6.3.0
I understand your confusion on the blog post you mentioned, though I have a feeling the intent of the writer was to more show how the JFrog CLI can be used.
To answer your questions:
1) Yes, you don't have to use the JFrog CLI to build. Please check out the documentation on how to set up a remote repository for Go. This will guide you through setting up GitHub or GoCenter as a remote repository for your Go builds. This will allow you to set the GOPROXY environment variable following this structure <protocol>://<username>:<password>#<artifactory domain>/api/go/<go repository>.
2) Remote repositories will absolutely act as a proxy, caching the contents you download from the remote repository (copying a part from the user guide: A remote Go repository in Artifactory serves as a caching proxy for a public Go registry such as GoCenter or GitHub.)
I had jcenter repository in my artifactory under remote repositories. Since some of the artifacts were missing such as qpid, I decided to delete jcenter and add it back. When I deleted the jcenter repository and added it back it didn't download any artifacts.
The repository tree structure looks like this
Before removing the jcenter repository it was like this.
Why it is not able to import any artifact? I can see the URL which is associated with it(http://jcenter.bintray.com/) has so many artifacts.
A remote repository in Artifactory serves as a caching proxy. This means that it downloads artifacts from the remote URL and cache them in Artifactory,
When you deleted the JCenter repository from Artifactory you deleted all cached artifacts.
After recreating the repository, your cache is empty. This is why, when browsing jcenter-cache, you see no artifacts. You can use the remote browsing capability in order to see which artifacts are available in the remote URL, but not currently caches
To re-populate the cache, you will need to download artifacts from the remote repository. Usually the best way to do it, is running your builds which are using this repository.
If the problem is that artifacts are not resolved at all from the remote repository, try the following:
Make sure the repository is configured correctly in Artifactory. Use the "Test" button to make sure that the URL is correct and you can reach the remote URL.
Check that your build tool is properly configured to use the repository you configured. One way of checking this, is by monitoring the Artifactory request log looking for requests from your build tool.
Deleting a repository is not a good practice when you are missing some dependencies. A better approach would be checking if they are available in the remote URL and downloading them into the cache. Artifactory has the option to perform a remote search in Bintray which can help you when looking for artifacts in JCenter.
When i create a maven prject which depend smtp 1.5.0, during updating denpendencies,
but the version exsit in center repo,
below is center repository config in my nexus(v2.2-01),
after do the operate (expire cache, repair index,update index)below,
Updating the project dependency again, but no use.
i check the storage browser,index browser, remote browser,
it seems my nexus has not get the index from center, and also has not get the smtp jar.
so what happened to my nexus?
I have small open-source projects hosted on Github which I want to make available for others via Maven. I have a small webspace where I can host static files. How can I create a repo? Also, I would want to remove old snapshots from there if possible.
Standard maven repository implementations are almost all Tomcat web apps. Each one of them should have a static repository, just as your local repository. The webapp serves to the purpose of searching and management of the artifacts stored in that static repository.
If you want to host the repository with static web access only, you'll have to perform the management manually and provide a static manually generated html page that contains GAV coordinates of all artifacts in the repo. No other user but you could ever upload to the repository unless you give your password or enable anonymous FTP acces.
If maven doesn't try to upload anything to the repo until the deploy phase then this approach is still partly usable, since running a mvn clean deploy should fail.
You can check if is it doable like this (I suppose that you have that projects in your local repo):
upload your local repoistory folder to a URL
for the purpose of testing mirror your central repo to that URL
try to build your project with dependencies from your repo
Open your settings.xml file and under <mirrors> node add:
<mirror>
<url>http://your/url/repo</url>
<mirrorOf>*</mirrorOf>
</mirror>
and see if mvn clean install suceeds. Please feedback.
In this SO answer I have outlined the way I set up my OSS projects which are all hosted in Github. There are actually a number of free services out there you could you when you would like to run an OSS project.
I would recommend publishing to Maven Central, if your plugin is well-tested and expected to bring other people benefits as well. You can use CloudBee's BuildHive as a free Jenkins CI.
A static repo works great, per my experience.
I scp'd up my local repository into a static apache server. Legit repo. Not as easy to maintain as a real repo of course, but quite a bit cheaper if you've already got a plain vanilla web host.
Other than setting the permissions properly (same as required for you to browse the folders), it was a pretty painless procedure.
The only two things I did to make it more reasonable were
1 - Wrote a script to "rm -rf ...." on most of the contents of my local repo so that the only thing I am deploying is those few artifacts that are not available in the general repos.
2 - Tarred it up first before scping to my web host.
Hope this helps.
The guy below did something similar, only using FTP which saves him a lot of hand work if he updates his binaries very often.
http://stuartsierra.com/2009/09/08/run-your-own-maven-repository
I think I know how to do it now. I'm using mvn deploy now to create a local repository on the file system and then I upload it to the webserver. If I'm not wrong, there doesn't even need to be a file listing.
The command I'm using is:
mvn deploy -DaltDeploymentRepository=local::default::file:./repo
This creates/updates the local repository automatically, so the repo can be synced with a server.
I work for a large company where we use an internally installed maven proxy for downloading dependencies. We have to use a proxy as our Network(Websense) does not allow downloading of files with the .jar extension. So whenever a pom declares an external dependency, maven tries to the get the jar files from the proxy. If it is not there in the proxy, the proxy will itself download it from the internet as the proxy is outside the purview of this Websense Nonsense. This method while not perfect works by and large.
The things that don't work are
Whenever we need to include a dependency for which there is no repository declared in the proxy server, that repository url has to be included manually.
The internal maven proxy does not have enough bandwidth to scale up. Some days it takes 10 minutes to download 7 MB jar file.
As a workaround we were thinking that if it were possible to install a P2P plugin in maven, then whenever a jar file is required maven could first check peers (for example local maven repo of the developer sitting next to me) before hitting the internal maven proxy. This will dramatically reduce the time it takes to download dependencies and also potentially avoid the bureaucracy of having someone to put an email to manually add maven repository urls.
You should install a repository manager (Artifactory, Archiva, Nexus in alphabetical order). The intention is to a have a single point which access the internet via the proxy and that should solve the problem. All developers only access the repository manager.