C++ community plugin: Sonar-runner scans wrong file types + Lexical errors - sonarqube

Question

I have setup a Sonar server and am trying to pull some metrics by setting up a test project, but I encounter some problems.
Environment:
OS: Windows Server 2008 R2 / amd64 / 6.1
App Server:Tomcat/7.0.40I, deployed the Sonar WAR file (3.5.1).
DB: MySQL v5.6.11
(relevant) Plugins:
Sonar C++ Community Plugin (0.2)
The plugin is visible under General Settings --> Sonar C++ Community Plugin
sonar-project.properties:
# Required metadata
sonar.projectKey=mistral:pmc
sonar.projectName=PMC
sonar.projectVersion=1.0
sonar.language=c++
# Comma-separated paths to directories with sources (required)
sonar.sources=e:/Development/mistral/PMC/trunk/
# Optional path to the CppCheck program required to activate some CppCheck rules
sonar.cpp.cppcheck.path=C:/Program Files (x86)/Cppcheck/cppcheck.exe
# Encoding of the source files
sonar.sourceEncoding=UTF-8
Issue 1:
When running the sonar-runner it fails as it tries to analyse irrelevant filetypes, like .DOC (see below)
C:\Users\Administrator\Documents\sonar-runner-2.2.1
Sonar Runner 2.2.1
Java 1.7.0_21 Oracle Corporation (64-bit)
Windows Server 2008 R2 6.1 amd64
INFO: Runner configuration file: C:\Users\Administrator\Documents\sonar-runner-2.2.1\conf\sonar-runner.properties
INFO: Project configuration file: C:\Users\Administrator\Documents\sonar-projects\PMC\sonar-project.properties
INFO: Default locale: "en_US", source code encoding: "UTF-8"
INFO: Work directory: C:\Users\Administrator\Documents\sonar-projects\PMC\.sonar
INFO: Sonar Server 3.5.1
13:18:23.320 INFO - Load batch settings
13:18:23.429 INFO - User cache: C:\Users\Administrator\.sonar\cache
13:18:23.444 INFO - Install plugins
13:18:24.287 INFO - ------------- Executing Project Scan
13:18:24.848 INFO - Install JDBC driver
13:18:24.848 INFO - Apply project exclusions
13:18:24.864 INFO - Create JDBC datasource for jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8
13:18:25.129 INFO - Initializing Hibernate
13:18:27.563 INFO - ------------- Inspecting PMC
13:18:27.578 INFO - Load module settings
13:18:28.780 INFO - Quality profile : [name=Default C++ Profile,language=c++]
13:18:28.795 INFO - Excluded tests:
13:18:28.795 INFO - **/package-info.java
13:18:28.826 INFO - Configure Maven plugins
13:18:28.982 INFO - Compare to previous analysis (2013-05-30)
13:18:29.045 INFO - Compare over 5 days (2013-05-25, analysis of 2013-05-29 09:23:04.0)
13:18:29.092 INFO - Compare over 30 days (2013-04-30, analysis of 2013-05-29 09:23:04.0)
13:18:29.294 INFO - Base dir: C:\Users\Administrator\Documents\sonar-projects\PMC
13:18:29.294 INFO - Working dir: C:\Users\Administrator\Documents\sonar-projects\PMC\.sonar
13:18:29.294 INFO - Source dirs: e:\Development\mistral\PMC\trunk
13:18:29.294 INFO - Source encoding: UTF-8, default locale: en_US
13:18:29.560 INFO - Sensor CxxSourceImporter...
13:18:30.932 WARN - SQL Error: 1366, SQLState: HY000
13:18:30.932 ERROR - Incorrect string value: '\xF2\x97\xA4\xB9\xDC\xAE...' for column 'data' at row 1
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
Total time: 8.408s
Final Memory: 12M/352M
INFO: ------------------------------------------------------------------------
ERROR: Error during Sonar runner execution
ERROR: Unable to execute Sonar
ERROR: Caused by: Unable to read and import the source file : 'e:\Development\mistral\PMC\trunk\doc\guides\Mistral_PMC_BroadcastEnvironmentImporter_user_guide.doc' with the charset : 'UTF-8'.
ERROR: Caused by: Unable to persist : SnapshotSource[snapshot_id=1742,data=??◄?→? > ♥ ?? ♠ ♀ ?♣ ► ?♣ ☺ ???? ?♣ ?♣ ?♣ ?♣ ?♣ ?♣ ?♣ ?♣ ?♣ ?♣ ?♣ ?♣ ?
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? g? ♦ ?↕?
??☼ ??☼ ??☼ ? ?§ ?§ ?" ?" 0# 0# 0# $ ???? T# T# T# P ?# ? ($ ? T# ?? ?☺ ?$ ?☻ ?' ( ?'
?' ?' ?+ j♦ ►0 l☺ |1 ? (? ☻ *? *? *? *? *? *? $ ?? ?☻ =? ?☺ N? § 0# 42 H+ H+ ^ ...]
ERROR: Caused by: org.hibernate.exception.GenericJDBCException: could not insert: [org.sonar.api.database.model.SnapshotSource]
ERROR: Caused by: could not insert: [org.sonar.api.database.model.SnapshotSource]
ERROR: Caused by: Incorrect string value: '\xF2\x97\xA4\xB9\xDC\xAE...' for column 'data' at row 1
ERROR:
ERROR: To see the full stack trace of the errors, re-run Sonar Runner with the -e switch.
ERROR: Re-run Sonar Runner using the -X switch to enable full debug logging.
When I forcefully exclude the offending file types by adding: sonar.exclusions=**/*.doc,**/*.svc to the sonar-project.properties file I can work around this problem, but I rather get it to work as expected (only take files into account as configured)
Issue 2:
The analysis throws allot of “Lexical error in file” errors while scanning (see below). I believe this might happen if the source encoding differs.
C:\Users\Administrator\Documents\sonar-projects\PMC>sonar-runner
C:\Users\Administrator\Documents\sonar-runner-2.2.1
Sonar Runner 2.2.1
Java 1.7.0_21 Oracle Corporation (64-bit)
Windows Server 2008 R2 6.1 amd64
INFO: Runner configuration file: C:\Users\Administrator\Documents\sonar-runner-2.2.1\conf\sonar-runner.properties
INFO: Project configuration file: C:\Users\Administrator\Documents\sonar-projects\PMC\sonar-project.properties
INFO: Default locale: "en_US", source code encoding: "UTF-8"
INFO: Work directory: C:\Users\Administrator\Documents\sonar-projects\PMC\.sonar
INFO: Sonar Server 3.5.1
13:22:35.048 INFO - Load batch settings
13:22:35.154 INFO - User cache: C:\Users\Administrator\.sonar\cache
13:22:35.157 INFO - Install plugins
13:22:35.878 INFO - ------------- Executing Project Scan
13:22:36.471 INFO - Install JDBC driver
13:22:36.476 INFO - Apply project exclusions
13:22:36.481 INFO - Create JDBC datasource for jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8
13:22:36.730 INFO - Initializing Hibernate
13:22:38.990 INFO - ------------- Inspecting PMC
13:22:38.995 INFO - Load module settings
13:22:40.078 INFO - Quality profile : [name=Default C++ Profile,language=c++]
13:22:40.096 INFO - Excluded sources:
13:22:40.097 INFO - **/*.doc
13:22:40.097 INFO - **/*.svc
13:22:40.097 INFO - Excluded tests:
13:22:40.098 INFO - **/package-info.java
13:22:40.125 INFO - Configure Maven plugins
13:22:40.266 INFO - Compare to previous analysis (2013-05-30)
13:22:40.326 INFO - Compare over 5 days (2013-05-25, analysis of 2013-05-29 09:23:04.0)
13:22:40.366 INFO - Compare over 30 days (2013-04-30, analysis of 2013-05-29 09:23:04.0)
13:22:40.575 INFO - Base dir: C:\Users\Administrator\Documents\sonar-projects\PMC
13:22:40.575 INFO - Working dir: C:\Users\Administrator\Documents\sonar-projects\PMC\.sonar
13:22:40.576 INFO - Source dirs: e:\Development\mistral\PMC\trunk
13:22:40.576 INFO - Source encoding: UTF-8, default locale: en_US
13:22:40.902 INFO - Sensor CxxSourceImporter...
13:22:42.888 INFO - Sensor CxxSourceImporter done: 1986 ms
13:22:42.889 INFO - Sensor CpdSensor...
13:22:42.889 INFO - SonarBridgeEngine is used
13:22:42.985 INFO - Cross-project analysis disabled
net.sourceforge.pmd.cpd.cppast.TokenMgrError: Lexical error in file e:\Development\mistral\PMC\trunk\AppServer\ApplicationDistributionScheduler\Logic\ActivateItvApplicationOnMcmDistributionRequestP
essor.cs at line 1, column 1. Encountered: "\ufeff" (65279), after : ""
at net.sourceforge.pmd.cpd.cppast.CPPParserTokenManager.getNextToken(CPPParserTokenManager.java:1655)
at net.sourceforge.pmd.cpd.CPPTokenizer.tokenize(CPPTokenizer.java:26)
at org.sonar.duplications.internal.pmd.TokenizerBridge.chunk(TokenizerBridge.java:60)
at org.sonar.duplications.internal.pmd.TokenizerBridge.chunk(TokenizerBridge.java:52)
at org.sonar.plugins.cpd.SonarBridgeEngine.analyse(SonarBridgeEngine.java:102)
at org.sonar.plugins.cpd.CpdSensor.analyse(CpdSensor.java:80)
at org.sonar.batch.phases.SensorsExecutor.execute(SensorsExecutor.java:70)
at org.sonar.batch.phases.Phases.execute(Phases.java:101)
at org.sonar.batch.scan.ScanContainer.doStart(ScanContainer.java:147)
at org.sonar.batch.bootstrap.Container.start(Container.java:72)
at org.sonar.batch.scan.ScanTask.scan(ScanTask.java:65)
at org.sonar.batch.scan.ScanTask.scanRecursively(ScanTask.java:56)
at org.sonar.batch.scan.ScanTask.execute(ScanTask.java:49)
at org.sonar.batch.bootstrap.TaskContainer.doStart(TaskContainer.java:188)
at org.sonar.batch.bootstrap.Container.start(Container.java:72)
at org.sonar.batch.bootstrap.TaskBootstrapContainer.executeTask(TaskBootstrapContainer.java:73)
at org.sonar.batch.bootstrap.TaskBootstrapContainer.doStart(TaskBootstrapContainer.java:62)
at org.sonar.batch.bootstrap.Container.start(Container.java:72)
at org.sonar.batch.bootstrap.BootstrapContainer.doStart(BootstrapContainer.java:104)
at org.sonar.batch.bootstrap.Container.start(Container.java:72)
at org.sonar.batch.bootstrapper.Batch.startBatch(Batch.java:88)
at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:72)
at org.sonar.runner.batch.IsolatedLauncher.execute(IsolatedLauncher.java:45)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.sonar.runner.impl.BatchLauncher$1.delegateExecution(BatchLauncher.java:75)
at org.sonar.runner.impl.BatchLauncher$1.run(BatchLauncher.java:63)
at java.security.AccessController.doPrivileged(Native Method)
at org.sonar.runner.impl.BatchLauncher.doExecute(BatchLauncher.java:57)
at org.sonar.runner.impl.BatchLauncher.execute(BatchLauncher.java:50)
at org.sonar.runner.api.EmbeddedRunner.doExecute(EmbeddedRunner.java:71)
at org.sonar.runner.api.Runner.execute(Runner.java:89)
at org.sonar.runner.Main.executeTask(Main.java:70)
at org.sonar.runner.Main.execute(Main.java:59)
at org.sonar.runner.Main.main(Main.java:41)
Skipping e:\Development\mistral\PMC\trunk\AppServer\ApplicationDistributionScheduler\Logic\ActivateItvApplicationOnMcmDistributionRequestProcessor.cs due to parse error
net.sourceforge.pmd.cpd.cppast.TokenMgrError: Lexical error in file e:\Development\mistral\PMC\trunk\AppServer\ApplicationDistributionScheduler\Logic\ApplicationDistributionRequestProcessor.cs at l
1, column 1. Encountered: "\ufeff" (65279), after : ""
at net.sourceforge.pmd.cpd.cppast.CPPParserTokenManager.getNextToken(CPPParserTokenManager.java:1655)
at net.sourceforge.pmd.cpd.CPPTokenizer.tokenize(CPPTokenizer.java:26)
at org.sonar.duplications.internal.pmd.TokenizerBridge.chunk(TokenizerBridge.java:60)
at org.sonar.duplications.internal.pmd.TokenizerBridge.chunk(TokenizerBridge.java:52)
at org.sonar.plugins.cpd.SonarBridgeEngine.analyse(SonarBridgeEngine.java:102)
at org.sonar.plugins.cpd.CpdSensor.analyse(CpdSensor.java:80)
at org.sonar.batch.phases.SensorsExecutor.execute(SensorsExecutor.java:70)
at org.sonar.batch.phases.Phases.execute(Phases.java:101)
at org.sonar.batch.scan.ScanContainer.doStart(ScanContainer.java:147)
at org.sonar.batch.bootstrap.Container.start(Container.java:72)
at org.sonar.batch.scan.ScanTask.scan(ScanTask.java:65)
at org.sonar.batch.scan.ScanTask.scanRecursively(ScanTask.java:56)
at org.sonar.batch.scan.ScanTask.execute(ScanTask.java:49)
at org.sonar.batch.bootstrap.TaskContainer.doStart(TaskContainer.java:188)
at org.sonar.batch.bootstrap.Container.start(Container.java:72)
at org.sonar.batch.bootstrap.TaskBootstrapContainer.executeTask(TaskBootstrapContainer.java:73)
at org.sonar.batch.bootstrap.TaskBootstrapContainer.doStart(TaskBootstrapContainer.java:62)
at org.sonar.batch.bootstrap.Container.start(Container.java:72)
at org.sonar.batch.bootstrap.BootstrapContainer.doStart(BootstrapContainer.java:104)
at org.sonar.batch.bootstrap.Container.start(Container.java:72)
at org.sonar.batch.bootstrapper.Batch.startBatch(Batch.java:88)
at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:72)
at org.sonar.runner.batch.IsolatedLauncher.execute(IsolatedLauncher.java:45)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.sonar.runner.impl.BatchLauncher$1.delegateExecution(BatchLauncher.java:75)
at org.sonar.runner.impl.BatchLauncher$1.run(BatchLauncher.java:63)
at java.security.AccessController.doPrivileged(Native Method)
at org.sonar.runner.impl.BatchLauncher.doExecute(BatchLauncher.java:57)
at org.sonar.runner.impl.BatchLauncher.execute(BatchLauncher.java:50)
at org.sonar.runner.api.EmbeddedRunner.doExecute(EmbeddedRunner.java:71)
at org.sonar.runner.api.Runner.execute(Runner.java:89)
at org.sonar.runner.Main.executeTask(Main.java:70)
at org.sonar.runner.Main.execute(Main.java:59)
at org.sonar.runner.Main.main(Main.java:41)
Skipping e:\Development\mistral\PMC\trunk\AppServer\ApplicationDistributionScheduler\Logic\ApplicationDistributionRequestProcessor.cs due to parse error
The thing is, I have set the sourceEncoding to UTF-8 but the files throwing the error are also encoded in UTF-8 (see below):
These might be simple beginner errors but I cannot find a quick/easy fix for this. Could you help on my way?

Answer 1

The .doc file is scanned because its file name ends with 'c'.
The c++ plugin's default source suffixes are: cxx,cpp,cc,c
Notice that there isn't any dot '.' character in there. So any file whose file name ends with 'c' character will be taken: abc, ab.c, a.bc. The same apply for the other suffixes.
If you want exact suffixes, you need to use something like this:
sonar.cxx.suffixes.sources=.cxx,.cpp,.cc,.c