OWASP Tricks to speed up vulnerabilities checks - maven

I use the maven project with the OWASP plugin to check the vulnerabilities on each commit in the CI pull requests.
The configuration looks pretty simple
<plugin>
<groupId>org.owasp</groupId>
<artifactId>dependency-check-maven</artifactId>
<version>${version.dependency-check-maven}</version>
<executions>
<execution>
<goals>
<goal>aggregate</goal>
</goals>
<phase>verify</phase>
</execution>
</executions>
<configuration>
<showSummary>true</showSummary>
<!-- this will work only in the top-level maven module -->
<suppressionFile>${user.dir}/owasp-suppressions.xml</suppressionFile>
<format>ALL</format>
<failBuildOnAnyVulnerability>true</failBuildOnAnyVulnerability>
<!-- alternative: fail on level (High starts at 7, Critical at 9) -->
<!-- <failBuildOnCVSS>4</failBuildOnCVSS> -->
<assemblyAnalyzerEnabled>false</assemblyAnalyzerEnabled>
<cveUrlModified>address of the NVD local mirror</cveUrlModified>
<cveUrlBase>address of the NVD local mirror</cveUrlBase>
<cveWaitTime>1</cveWaitTime> <!--value in milliseconds, default is 4000-->
</configuration>
</plugin>
For CI builds I use PROW - https://docs.prow.k8s.io/docs/overview/
For local mirror approach is described here https://jeremylong.github.io/DependencyCheck/data/mirrornvd.html and the mirror is used from here https://github.com/stevespringett/nist-data-mirror/
Still the process to download CVEs from mirror takes quite a lot of time (around 3 minutes). From the logs I can see the most of time is spent on downloading CVEs
[INFO] Download Started for NVD CVE - 2003
[INFO] Download Complete for NVD CVE - 2003 (5 ms)
[INFO] Processing Started for NVD CVE - 2003
[INFO] Processing Complete for NVD CVE - 2002 (4608 ms)
[INFO] Processing Complete for NVD CVE - 2003 (1131 ms)
...
and for doing some DB maintenance?
[INFO] Begin database maintenance
[INFO] Updated the CPE ecosystem on 128773 NVD records
[INFO] Removed the CPE ecosystem on 3604 NVD records
[INFO] End database maintenance (13482 ms)
[INFO] Begin database defrag
[INFO] End database defrag (3765 ms)
[INFO] Check for updates complete (112132 ms)
Do you know any tricks to speed up OWASP checks?

This was replied on this other thread: How to cache OWASP dependecy check NVD database on CI
Basically you need to tell PROW to cache the location of the NVD database which when using the Maven plugin is:
$MAVEN_HOME/.m2/repository/org/owasp/dependency-check-data/7.0/nvdcache/
** Considering that you use Dependency-Check version 7.+

Related

Using the spotbugs maven plugin, where do I put filter files?

I am trying to use spotbugs from maven.
In the <reporting/> section of my POM, I include
<plugin>
<groupId>com.github.spotbugs</groupId>
<artifactId>spotbugs-maven-plugin</artifactId>
<version>4.5.3.0</version>
<configuration>
<includeFilterFile>spotbugs-security-include.xml</includeFilterFile>
<excludeFilterFile>spotbugs-security-exclude.xml</excludeFilterFile>
<plugins>
<plugin>
<groupId>com.h3xstream.findsecbugs</groupId>
<artifactId>findsecbugs-plugin</artifactId>
<version>1.11.0</version>
</plugin>
</plugins>
</configuration>
</plugin>
Where do I put the filter files spotbugs-security-include.xml and spotbugs-security-exclude.xml?
According to the documentation, the plugin will find the filter files on the classpath. But what is the classpath for executing a reporting plugin? And where in the source tree do I put the files in order to have them copied there during report generation? I have tried src/main/resources, src/test/resources, and src/site/resources with no luck.
Log snippet:
[INFO] 6 reports detected for maven-javadoc-plugin:3.2.0: aggregate-no-fork, javadoc, javadoc-no-fork, test-aggregate-no-fork, test-javadoc, test-javadoc-no-fork
[INFO] configuring report plugin com.github.spotbugs:spotbugs-maven-plugin:4.5.3.0
[INFO] 1 report detected for spotbugs-maven-plugin:4.5.3.0: spotbugs
[INFO] configuring report plugin org.apache.maven.plugins:maven-project-info-reports-plugin:3.1.2
[INFO] 15 reports detected for maven-project-info-reports-plugin:3.1.2: ci-management, dependencies, dependency-info, dependency-management, distribution-management, index, issue-management, licenses, mailing-lists, modules, plugin-management, plugins, scm, summary, team
[INFO] Fork Value is true
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 25.587 s
[INFO] Finished at: 2022-02-11T15:31:56+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Could not find resource 'spotbugs-security-include.xml'. -> [Help 1]
I run under OpenJDK 11 on linux.

Vaadin Custom Components/Widgets with Maven and Spring Boot

I am relatively new to vaadin and started out with a spring boot application and the vaadin spring boot plugin. Everything worked fine until I got to the point where I tried to create my own components/widgets.
Unfortunately I didn't find any "official" example/documentation how to set up custom components within a spring boot application so I had to search the web to find out how to set up additional plugin(s) in maven to compile the code for the client side widgets. As far as I can tell from the log output the compilation of these components work, but when I try to access these components on the webpage I get an error:
Widgetset 'com.vaadin.DefaultWidgetSet' does not contain implementation for net.gtidev.test.components.MyComponent. Check its component connector's #Connect mapping, widgetsets GWT module description file and re-compile your widgetset. [...]
Here is the widget compiler log:
[INFO] Using com.vaadin:vaadin-client-compiler version 7.6.4
[ERROR] Mar 22, 2016 10:22:43 AM java.util.prefs.WindowsPreferences <init>
[ERROR] WARNUNG: Could not open/create prefs root node Software\JavaSoft\Prefs at root 0x80000002. Windows RegCreateKeyEx(...) returned error code 5.
[INFO] Compiling module net.gtidev.test.components.TestWidgetset
[INFO] Computing all possible rebind results for 'com.vaadin.client.metadata.ConnectorBundleLoader'
[INFO] Rebinding com.vaadin.client.metadata.ConnectorBundleLoader
[INFO] Invoking generator com.vaadin.server.widgetsetutils.ConnectorBundleLoaderFactory
[INFO] Populating eager bundle
. . . . . 250 more lines
[INFO] Computing all possible rebind results for 'com.vaadin.client.ui.dd.VAcceptCriterionFactory'
[INFO] Rebinding com.vaadin.client.ui.dd.VAcceptCriterionFactory
[INFO] Invoking generator com.vaadin.server.widgetsetutils.AcceptCriteriaFactoryGenerator
[INFO] Detecting available criteria ...
[INFO] creating mapping for com.vaadin.event.dd.acceptcriteria.AcceptAll
[INFO] creating mapping for com.vaadin.event.dd.acceptcriteria.And
[INFO] creating mapping for com.vaadin.event.dd.acceptcriteria.ContainsDataFlavor
[INFO] creating mapping for com.vaadin.event.dd.acceptcriteria.SourceIs
[INFO] creating mapping for com.vaadin.ui.AbstractSelect.TargetItemIs
[INFO] creating mapping for com.vaadin.ui.AbstractSelect.AcceptItem
[INFO] creating mapping for com.vaadin.ui.Table.TableDropCriterion
[INFO] creating mapping for com.vaadin.ui.Tree.TreeDropCriterion
[INFO] creating mapping for com.vaadin.event.dd.acceptcriteria.Not
[INFO] creating mapping for com.vaadin.event.dd.acceptcriteria.Or
[INFO] creating mapping for com.vaadin.event.dd.acceptcriteria.ServerSideCriterion
[INFO] creating mapping for com.vaadin.event.dd.acceptcriteria.SourceIsTarget
[INFO] creating mapping for com.vaadin.event.dd.acceptcriteria.TargetDetailIs
[INFO] creating mapping for com.vaadin.ui.Tree.TargetInSubtree
[INFO] Done. (0seconds)
[INFO] Compiling 1 permutation
[INFO] Compiling permutation 0...
[INFO] Compile of permutations succeeded
[INFO] Compilation succeeded -- 59,217s
[INFO] Linking into C:\projects\misc\vaadin-boot\target\vaadin-boot-0.0.1-SNAPSHOT\net.gtidev.test.components.TestWidgetset
[INFO] Link succeeded
[INFO] Linking succeeded -- 0,492s
The files I use for my custom component were generated by the eclipse vaadin addon within a vaadin 7 project which I created only for this purpose. When I started this vaadin 7 project in eclipse the component worked. I then copied these files into my spring boot maven project where the custom component does not get loaded any more.
I know that spring boot applications have a slightly different bootstrap mechanism and layout than "classic" webapps and that amongst other things static resources are not loaded from the webapp-folder but from the classpath:/static folder. I think that the core of the problem has something to do with this fact but I don't know what to do to fix it.
My Plugin configuration (I tried with and without the commented options):
<plugin>
<groupId>com.vaadin</groupId>
<artifactId>vaadin-maven-plugin</artifactId>
<version>7.6.4</version>
<configuration>
<strict>true</strict>
<force>true</force>
<!-- Enable during development to speed compiling. -->
<!-- <draftCompile>true</draftCompile>
<style>DETAILED</style> -->
<!-- End development options -->
<!--<webappDirectory>src/main/webapp/VAADIN/widgetsets</webappDirectory>-->
<modules>
<module>net.gtidev.test.components.TestWidgetset</module>
</modules>
</configuration>
<executions>
<execution>
<goals>
<goal>resources</goal>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
I tried different maven plugin combinations and configurations. In one example, there was also a Google-GWT Plugin mentioned, but running this plugin on the code produced the same log output as the vaadin plugin:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>gwt-maven-plugin</artifactId>
<version>2.5.1</version>
<!--<configuration>-->
<!--<webappDirectory>src/main/webapp/VAADIN/widgetsets</webappDirectory>-->
<!--<extraJvmArgs>-Xmx512M -Xss1024k</extraJvmArgs>-->
<!--<runTarget>clean</runTarget>-->
<!--<hostedWebapp>${project.build.directory}/${project.build.finalName}</hostedWebapp>-->
<!--<noServer>true</noServer>-->
<!--<port>8080</port>-->
<!--<soycDetailed>false</soycDetailed>-->
<!--</configuration>-->
<executions>
<execution>
<goals>
<goal>resources</goal>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
To use custom client side extensions, Vaadin Add-ons, you'll need to add vaadin-maven-plugin to your project. It will scan the add-ons you use and GWT compile a new widgetset for your project that contains those extensions.
If you created the project with start.spring.io the maven plugin is not in your project by default. Create an example project for example using this Vaadin+Spring archetype or the official servlet based archetype and copy the vaadin-maven-plugin related parts from the pom.xml to your projects pom.xml. Then do a full build and everything should works as expected.

Jmeter Maven Plugin - Remote server config with parameters in pom.xml

I am trying to execute a jmeter test using the maven plugin on a remote server leveraging a pom.xml file that is completely parameterized as below.(I was able to successfully run a maven-jmeter remote test, where the servername, port, protocol and other info are all hard coded in the jmx file.
My pom.xml file is below
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>jmeter-demo</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>jmeter-demo</name>
<url>http://maven.apache.org</url>
<description>
Executes a JMeter test.
</description>
<properties>
<webapp.protocol>http</webapp.protocol>
<!--<webapp.host>www.mozilla.com</webapp.host>-->
<webapp.host>fsa-mia-dev2.fsalabs.io</webapp.host>
<webapp.port>80</webapp.port>
<test.duration>30</test.duration>
<test.threads>10</test.threads>
</properties>
<build>
<plugins>
<plugin>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-maven-plugin</artifactId>
<version>1.9.0</version>
<executions>
<execution>
<!--
1)
first test run warms up the webserver.
Used to fill caches.
With a different set of properties since it runs much shorter than a normal test
and also the rate of requests/second may be much lower.
Maybe also use a different URL set.
-->
<id>warmup</id>
<phase>integration-test</phase>
<goals>
<goal>jmeter</goal>
</goals>
<configuration>
<propertiesUser>
<!--Accesses urls for warmup (compiles JSPs, fills caches, ...)-->
<!--************Setup Config for Threadgroup 23***************-->
<threadgroup23.name>S23_SFA_Org_CO</threadgroup23.name>
<threadgroup23.Transactionname>T23_SFA_Org_CO</threadgroup23.Transactionname>
<threadgroup23.ThinkTime>5000</threadgroup23.ThinkTime>
<threadgroup23.comment>T23_SFA_Org_CO</threadgroup23.comment>
<!--number of threads to use-->
<threadgroup23.numberOfThreads>5</threadgroup23.numberOfThreads>
<!--delay of the test in seconds-->
<threadgroup23.scheduledDelay>0</threadgroup23.scheduledDelay>
<!--duration of the test in seconds-->
<threadgroup23.scheduledDuration>60</threadgroup23.scheduledDuration>
<!--how long till all threads are up and running in seconds-->
<threadgroup23.rampUp>1</threadgroup23.rampUp>
<threadgroup23.dataFile>/home/ubuntu/mproj/data/S23_SFA_Org_CO.dat</threadgroup23.dataFile>
</propertiesUser>
</configuration>
</execution>
</executions>
<configuration>
<testFilesIncluded>
<testFilesIncluded>01_SA_CU_GetCPS.jmx</testFilesIncluded>
</testFilesIncluded>
<!-- protocol, server and port of tested webapp -->
<propertiesUser>
<protocol>${webapp.protocol}</protocol>
<server>${webapp.host}</server>
<port>${webapp.port}</port>
</propertiesUser>
<resultsFileFormat>csv</resultsFileFormat>
<testResultsTimestamp>false</testResultsTimestamp>
<ignoreResultFailures>true</ignoreResultFailures>
<remoteConfig>
<startServersBeforeTests>true</startServersBeforeTests>
<serverList>10.36.14.170</serverList>
</remoteConfig>
<suppressJMeterOutput>false</suppressJMeterOutput>
<propertiesSystem>
</propertiesSystem>
<propertiesJMeter><log_level.jmeter>DEBUG</log_level.jmeter></propertiesJMeter>
</configuration>
</plugin>
</plugins>
</build>
</project>
The issue is.. jmeter-server detects a request to execute and almost immediately stops.
Here is the info from log file post run
[debug] JMeter is called with the following command line arguments: -n -t /home/ubuntu/mProj/src/test/jmeter/01_SA_CU_GetCPS.jmx -l /home/ubuntu/mProj/target/jmeter/results/01_SA_CU_GetCPS.csv -d /home/ubuntu/mProj/target/jmeter -j /home/ubuntu/mProj/target/jmeter/logs/01_SA_CU_GetCPS.jmx.log -r -R 10.36.14.170
[info] Executing test: 01_SA_CU_GetCPS.jmx
[info] Created the tree successfully using /home/ubuntu/mProj/src/test/jmeter/01_SA_CU_GetCPS.jmx
[info] Configuring remote engine for 10.36.14.170
[info] Using remote object: UnicastRef [liveRef: [endpoint:[10.36.14.170:37913](remote),objID:[-10468c89:14fba100421:-7fff, -1854422229136130507]]]
[info] Starting remote engines
[info] Starting the test # Fri Sep 11 01:42:27 UTC 2015 (1441935747716)
[info] Remote engines have been started
[info] Waiting for possible shutdown message on port 4445
[info] Tidying up remote # Fri Sep 11 01:42:29 UTC 2015 (1441935749973)
[info] Completed Test: 01_SA_CU_GetCPS.jmx
[INFO]
[INFO] Test Results:
[INFO]
[INFO] Tests Run: 1, Failures: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 25.280s
[INFO] Finished at: Fri Sep 11 01:42:35 UTC 2015
[INFO] Final Memory: 11M/57M
[INFO] ------------------------------------------------------------------------
(The pom-file works well without the remoteConfig - in localhost)
(There's no networking issues any where and the remote server is able to access the host and works when servername, port, protocol is hardcoded in jmx files)
Is this a known limitation of the maven-jmeter plugin?
First use a recent version of jmeter-maven-plugin, 2.8.3 as of 15th january 2019.
To pass parameters, enter in pom.xml as child of configuration:
<propertiesJMeter>
<BUILD_TAG>${project.version}</BUILD_TAG>
<threads>7</threads>
<duration>30</duration>
</propertiesJMeter>
And in jmeter, to use threads for example, you would use __P function:
${__P(threads,5)
This blog explains the whole process.

nexus-staging-maven-plugin refuse to upload after deferred deployment

I have a multi-module project that has nexus-staging-maven-plugin configured in parent pom.xml:
<plugin>
<groupId>org.sonatype.plugins</groupId>
<artifactId>nexus-staging-maven-plugin</artifactId>
<version>1.6.6</version>
<extensions>true</extensions>
<configuration>
<serverId>ossrh</serverId>
<nexusUrl>https://oss.sonatype.org/</nexusUrl>
<!--<autoReleaseAfterClose>true</autoReleaseAfterClose>-->
</configuration>
</plugin>
By default it should be inherited by all submodules (except those disabled using technique in How to disable nexus-staging-maven-plugin in sub-modules)
However, when I start deployment:
mvn clean deploy -DskipTests=true -Prelease-sign-artifacts -Dgpg.passphrase=*****
I see the following message:
[INFO] Installing Nexus Staging features:
[INFO] ... total of 5 executions of maven-deploy-plugin replaced with nexus-staging-maven-plugin
...
[INFO] --- nexus-staging-maven-plugin:1.6.6:deploy (injected-nexus-deploy) # spookystuff-core ---
[INFO] Performing deferred deploys (gathering into "/home/peng/git/spookystuff/target/nexus-staging/deferred")...
[INFO] Installing /home/peng/git/spookystuff/core/target/spookystuff-core-0.3.2-SNAPSHOT.jar to /home/peng/git/spookystuff/target/nexus-staging/deferred/com/tribbloids/spookystuff/spookystuff-core/0.3.2-SNAPSHOT/spookystuff-core-0.3.2-SNAPSHOT.jar
...
[INFO] Reactor Summary:
...
[INFO] BUILD SUCCESS
No upload happens whatsoever. The artifact that should be uploaded to nexus were still cached under:
/target/nexus-staging/deferred
but neither the log nor nexus server record indicates that it has been uploaded. What has been wrong here and what should I do to fix it?

Selenium tests failing in Unix

I am trying to integrate Selenium tests in Bamboo. I am able to invoke the tests on a Windows box but when I try to replicate the same on the Unix server I get the following error:
*rror 29-Aug-2012 17:03:54 17:03:54.792 WARN - Caution: '/usr/bin/google-chrome': file is a script file, not a real executable. The browser environment is no longer fully under RC control
build 29-Aug-2012 17:03:54 17:03:54.803 INFO - Launching Google Chrome...
build 29-Aug-2012 17:33:55 17:33:55.319 INFO - Killing Google Chrome...
build 29-Aug-2012 17:33:57 [INFO] ------------------------------------------------------------------------
build 29-Aug-2012 17:33:57 [INFO] BUILD FAILURE
build 29-Aug-2012 17:33:57 [INFO] ------------------------------------------------------------------------
build 29-Aug-2012 17:33:57 [INFO] Total time: 33:21.497s
build 29-Aug-2012 17:33:57 [INFO] Finished at: Wed Aug 29 17:33:57 BST 2012
build 29-Aug-2012 17:33:57 [INFO] Final Memory: 28M/164M
build 29-Aug-2012 17:33:57 [INFO] ------------------------------------------------------------------------
build 29-Aug-2012 17:33:57 [ERROR] Failed to execute goal org.codehaus.mojo:selenium-maven-plugin:2.3:selenese (test) on project iopscience-webapp: Execution test of goal org.codehaus.mojo:selenium-maven-plugin:2.3:selenese failed. SeleniumCommandTimedOutException -> [Help 1]*
I am using the selenium-maven-plugin to run the selenese tests suits. Following is the extract of my pom.xml
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>selenium-maven-plugin</artifactId>
<version>2.3</version>
<configuration>
<browser>*googlechrome</browser>
<suite>Testsuite-UI.html</suite>
<startURL>http://localhost:8082/</startURL>
</configuration>
<executions>
<execution>
<id>test</id>
<phase>integration-test</phase>
<goals>
<goal>selenese</goal>
</goals>
</execution>
</executions>
</plugin>
Please let me know if there is any specific configuration to be done on Linux to run the test suites?

Resources