Within my organisation, I'm heavily regulated by an IT security body. As such, all access to maven mirrors is prohibited. However, I can provide details of the dependencies I require so that the IT security bods can perform their checks / scans, and hopefully download the binaries for me.
I then install these in my local maven repo, which means that they're freely available to my team inside the firewall.
The beauty of maven is that it can identify and download transitive dependencies. However, without being able to access external POM files, I cannot perform these checks via maven dependency:tree.
As a result, I have to provide the dependencies one-at-a-time to the IT department. Given the depth of some of the dependency chains, this can take quite some time to go back/forward with IT. Alternatively, I do the dependency:tree at home, and mail myself the results.
I was wondering if anyone had any suggestions of websites where I can query for this information? So if I want (say) log4j:1.2.17, it will tell me that I also need to ask for javax.mail:mail:1.4.3 too.
Unfortunately I'm not aware of any sites that give you the full transitive dependency information, but there is a good solution to the underlying problem.
Namely, you can try to convince management to use a repository manager with an artifact auditing feature.
Sonatype Nexus certainly has this (see "Artifact Procurement"), JFrog Artifactory seems to enable this as well (through License Control"), both as extras in paid editions.
Such an approach will save time and money in the long run. Repository managers are much less fragile than a shared local repo:
they are an explicit entity intended for this purpose (and Maven has specific settings explicitly intended for working with them),
they have dedicated authentication/authorization mechanisms that can be made to work with existing ones in your company,
you can use the deploy plugin with them to properly publish build artifacts,
etc. etc.
Another relevant reason that may sway over your security body is that you're be able to use the same auditing procedure both for external and internal artifacts.
Related
Why do some companies or projects host nexus on their own domain instead of using Maven Central Repository? Is it related to security? Is this good practice?
Several reasons:
Have a place for the artifacts you build in your company.
Proxy several external repositories, so that the settings.xml only needs to have an entry for the mirror.
Circumvent proxy/firewall problems that stop developers from using outside repositories directly.
Actually, if you have more than one or two developers, it is the way to go.
In addition to proxying several external repositories already mentioned, repository manager groups allow you to combine hosted repositories as well into a single source.
Said group can then be organized to return components in an ordered manner (central first, then others, for example).
This makes configuration simpler and allow you to access your internal and external stuff from one place.
Additionally, if you wish to restrict who can access what, you can setup security policies to this affect. Usually that's not just related to Central, but imagine if you had 3 teams and didn't want them to share each others artifacts. Then 3 repositories, restrict the security per team but can be same config.
Another benefit is caching. If you download something, I download something and JF downloads something, that's 3 hits to the internet. If NXRM downloads it, that's 1 hit then you have it in your intranet.
Note, pretty much everything I just said is not related to just Maven, they're general repository manager perks.
In the AP Computer Science class up at my school, we get a good amount of time to work on individual coding projects.
This is great and all, but the school's proxy blocks dependency handlers such as Maven and Gradle.
I'm working on a fairly large project that requires Maven to handle its dependencies, and I'd really like to be able to work on it at school.
Now, I've talked to the IT guy and some of the administrators at my school and they've all said to submit a ticket with specific URLs to our Tech Admin, and they'll [quickly] submit it to the people who manage the blacklist and have them remove it.
My problem is that I don't know the specific URLs for them. The websites for the tools themselves aren't blocked, but the service (the act of the programs doing what they're supposed to) is.
These are default repositories for maven and gradle (as of oct 17 with their latest releases):
Maven Central - https://repo.maven.apache.org/maven2/
Gradle (jcenter) - https://jcenter.bintray.com/
I have posted several java libraries to Bintray, and then linked them to Jcenter. For the sake of argument, let's call one of the libraries 'my.private:repo'. This enables me to use the library in Gradle like this:
...
repositories {
jcenter()
}
...
dependencies {
...
implementation 'my.private:repo:1.0.0'
...
}
Excellent.
Now, a perhaps irrational fear of mine is that I will hit the bandwidth or storage limit for my Bintray repos, and so I will have to start paying to host the repos on Bintray (currently I have 18 repos on the open source plan). My question is whether it is possible to host the libraries somewhere else (like my own private server), and then have Jcenter simply do the redirection. I am fully aware that I COULD set up a private server and then do something like this...
repositories {
jcenter()
maven { url ... }
}
But I really don't want to do this. I want to remove as much friction as possible from the build process...which means relying on a single server to do the redirection (like a DNS), and another server to do the actual hosting.
So is it possible to host the libraries myself, and simply have Jcenter do the redirection so that I can stick with the first code snippet?
Ideas?
As an aside, it seems like it would be smart to decouple the two things (redirection and hosting), but I understand that it is more profitable for Bintray to couple them since they can essentially force everybody who steps out of the memory or bandwidth limits to pay (which they couldn't do if they were simply do the redirection).
I am not an expert on JCenter. However they could not possible offer such a feature for the billions of security concerns this would impose to them.
Imagine, you would be able to become a "backend repository" for JCenter (which would host maven artefacts on it's own). in case jcenter would not know a specific artefact on it's own, it would come and ask you about it. This would be a great vector for fraudsters to become a jcenter backend and publish "patched" libraries containing any sort of nasty code. In this case, JCenter would become distributor of modified (and possibly dangerous) artifacts from third parties.
The security threat would also exist if you would just ask for a redirect to a specific address. that server of yours could still become taken over by hackers and a modified jar could be published.
In both cases, it would be JCenter distributing possibly modified jar files with next to no control of the contents.
as an alternative - consider uploading your artefacts within the build job to a more durable maven repository - such as maven central.
I read articles from 3 years back highlighting why Jcenter is a better option over Maven Central, one of the primary reasons being the HTTPS transfer as a result of distribution via a CDN. Maven Central had HTTP, does the Maven repository still use HTTP?
It would make sense for them to upgrade to HTTPS, matching their competition (despite other areas where Jcenter excels). Have they done this yet?
Additionally, I'd like to ask why using a CDN enhances security. How does Maven Central distribute its libraries if not via a CDN?
Many thanks in advance.
HTTPS and CDN are two orthogonal concerns of a distribution repository, and JCenter and Maven Central have both.
We (JFrog) did many experiments with CDNs (including Fastly, that Maven Central uses), and went with the widest and fastest there is on the market today – Akamai. It means that your downloads will be served from servers closer to you, and the downloads will be faster.
But the differences aren't limited to CDN and presence or lacking of HTTPS. For example, the existence of additional metadata, like the maintainer, release notes, license information, and more is critical for a modern distribution repository, and the ability to be the owner of your package while making it available via 'canonical' repositories is also a huge advantage for open-source developers.
I am with JFrog, the company behind bintray, jcenter and artifactory, see my profile for details and links.
I am reading Repository Management with Nexus and the focus of it seems to act as a local proxy. Instead I would like to use it to distribute custom artifacts (very few of them, like less than 10). Some of them might be open source and some private to a company or another company (I'm a consultant).
Before I read the whole book and find out that Nexus is not for me, do you think this is a reasonable use case? I'm only at chapter 2, so I don't know what kind of authorization can Nexus provide for a single artifact. One option would be to install multiple copies of Nexus in different path, with http password I guess, albeit probably not the smartest.
The purpose of this question is to know if Nexus is suitable to distribute private artifacts to different companies with different privileges and to work on the internet, not in a intranet, or I should look for other options. Thanks!
I think this is a very reasonable thing to ask of a Nexus installation. I've used both Artifactory and Nexus (2 of the most popular Maven repos), and found Nexus to be much more flexible and full-featured. Sonatype has a similar setup to what you are asking about for their open source artifact hosting. In that case, I believe the security only restricts uploads, not downloads, but I'm 99% certain that downloads can also be secured. JBoss also has a large public Nexus installation.
Do keep in mind, though, that Sonatype has both an OSS edition and a commercial edition of Nexus. The segregation you're looking for may be a commercial feature only. But I would still recommend Nexus for the purpose you described, as long as the cost isn't prohibitive. Hint: I think you'll start to really find what you're looking for when you get to chapter 6.
Nexus allows you to have both hosted and proxy repositories. Apart from this it allows you to have virtual repositories and groups. Groups can be used for grouping your repositories under one name. So... you can set up a repository containing some artifacts that should be visible only to your clients, another one for your OSS artifacts and then group them for some client.
You can also use the Pro version, which is paid, and, as far as I recall had this sort of feature.