This is my first time working in Spring. I am working on a webapp that will intake all needed information on one screen and sort out the data to its appropriate insert statements. I have 7 inserts that all correspond to their own respective tables but only 1 model file. I was wondering if having all information in one model is possible, and if so, how can I implement this?
public class Metadata implements Serializable{
String app_nm
String schdlr_nm
String btch_nm
String dir_nm
String etl_nm
String enc
String file_nm
String schema_nm
String path
String tbl_nm
String ext
String obj_nm
....
}
scheduler_tbl - app_nm, schdlr_nm
batch_tbl - btch_nm, dir_nm
job_tbl - btch_nm, etl_nm
jobparam_tabl - btch_nm, enc, file_nm
dataobj_tbl - obj_nm, btch_nm, schema_nm
file_ptrn_tbl - path, tbl_nm, ext, obj_nm
optional_tbl - app_nm, btch_nm, obj_nm
Related
I am trying to dump data from csv to MySQL using Spring Batch. But on running the application, the following error is encountered:
org.springframework.batch.item.file.FlatFileParseException: Parsing error at line: 2 in resource=[class path resource [People2.csv]], input=[aardsda01,1981,12,27,USA,CO,Denver,2022,2,22,USA,NJ,Atlanta,David,Aardsma,David Allan,215,75,R,R,06-04-2004,23-08-2015,aardd001,aardsda01]
at org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:194) ~[spring-batch-infrastructure-5.0.0.jar:5.0.0]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.read(AbstractItemCountingItemStreamItemReader.java:95) ~[spring-batch-infrastructure-5.0.0.jar:5.0.0]
at org.springframework.batch.core.step.item.SimpleChunkProvider.doRead(SimpleChunkProvider.java:110) ~[spring-batch-core-5.0.0.jar:5.0.0]
at org.springframework.batch.core.step.item.SimpleChunkProvider.read(SimpleChunkProvider.java:189) ~[spring-batch-core-5.0.0.jar:5.0.0]
at org.springframework.batch.core.step.item.SimpleChunkProvider$1.doInIteration(SimpleChunkProvider.java:137) ~[spring-batch-core-5.0.0.jar:5.0.0]
...............
Caused by: org.springframework.batch.item.file.transform.IncorrectTokenCountException: Incorrect number of tokens found in record: expected 24 actual 7
at org.springframework.batch.item.file.transform.AbstractLineTokenizer.tokenize(AbstractLineTokenizer.java:133) ~[spring-batch-infrastructure-5.0.0.jar:5.0.0]
at org.springframework.batch.item.file.mapping.DefaultLineMapper.mapLine(DefaultLineMapper.java:42) ~[spring-batch-infrastructure-5.0.0.jar:5.0.0]
at org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:190) ~[spring-batch-infrastructure-5.0.0.jar:5.0.0]
... 41 common frames omitted
The ItemReader and ItemWriter Beans are:
#Bean
public FlatFileItemReader<Player> reader()
{
FlatFileItemReader<Player> reader=new FlatFileItemReader<Player>();
reader.setResource(new ClassPathResource("People2.csv"));
reader.setLineMapper(getLineMapper());
reader.setLinesToSkip(1);//Skip line in case of error
return reader;
}
private LineMapper<Player> getLineMapper() {
DefaultLineMapper<Player> lineMapper
=new DefaultLineMapper<Player>();
DelimitedLineTokenizer lineTokenizer=new DelimitedLineTokenizer();
lineTokenizer.setNames(new String[] {"playerID","birthYear","","","","","birthCity","","","","","","","nameFirst","nameLast","","","","","","","","retroID","bbrefID"});//name of field to be taken out. Empty string are columns which has to be skipped
lineTokenizer.setIncludedFields(new int[] {0,1,6,13,14,22,23});
BeanWrapperFieldSetMapper<Player> fieldSetMapper=new BeanWrapperFieldSetMapper<Player>();
fieldSetMapper.setTargetType(Player.class);
lineMapper.setLineTokenizer(lineTokenizer);
lineMapper.setFieldSetMapper(fieldSetMapper);
return lineMapper;
}
Spring Boot version I am using is 3.0.1
Incorrect number of tokens found in record: expected 24 actual 7 means you have less tokens in your input line than it is expected by your batch job.
In fact, you declared 24 names in lineTokenizer.setNames, but only included 7 in lineTokenizer.setIncludedFields. Those two parameters should match. You should think of it like: "Please include only these fields, and here are their names".
If you remove those empty "" (no need for them) and your input line has exactly 7 columns, then this error should not happen.
I have a method in our software that pulls the text from a PDF, from a scan or text generated.
I usually try the GetTextFromPage() method first. If it doesn't return text, then I move onto OCR'ing the page.
I have a particular 6 page PDF with the first three pages being a scanned document, and the last two being a form.
On this PDF I'm getting an error that I can't figure out how to resolve.
'StandardEncoding' is not a supported encoding name. For information on defining a custom encoding, see the documentation for the Encoding.RegisterProvider method.
Parameter name: name
at System.Globalization.EncodingTable.internalGetCodePageFromName(String name)
at System.Globalization.EncodingTable.GetCodePageFromName(String name)
at iText.IO.Util.IanaEncodings.GetEncodingEncoding(String name)
at iText.IO.Util.EncodingUtil.ConvertToBytes(Char[] chars, String encoding)
at iText.IO.Font.PdfEncodings.ConvertToBytes(String text, String encoding)
at iText.IO.Font.FontEncoding.FillNamedEncoding()
at iText.IO.Font.FontEncoding.CreateFontEncoding(String baseEncoding)
at iText.Kernel.Font.PdfType1Font..ctor(PdfDictionary fontDictionary)
at iText.Kernel.Font.PdfFontFactory.CreateFont(PdfDictionary fontDictionary)
at iText.Kernel.Pdf.Canvas.Parser.PdfCanvasProcessor.GetFont(PdfDictionary fontDict)
at iText.Kernel.Pdf.Canvas.Parser.PdfCanvasProcessor.SetTextFontOperator.Invoke(PdfCanvasProcessor processor, PdfLiteral operator, IList`1 operands)
at iText.Kernel.Pdf.Canvas.Parser.PdfCanvasProcessor.InvokeOperator(PdfLiteral operator, IList`1 operands)
at iText.Kernel.Pdf.Canvas.Parser.PdfCanvasProcessor.ProcessContent(Byte[] contentBytes, PdfResources resources)
at iText.Kernel.Pdf.Canvas.Parser.PdfTextExtractor.GetTextFromPage(PdfPage page, ITextExtractionStrategy strategy, IDictionary`2 additionalContentOperators)
at iText.Kernel.Pdf.Canvas.Parser.PdfTextExtractor.GetTextFromPage(PdfPage page)
at EFR.OCR.OCR.ExtractTextFromPDF(FileInfo fileInfo, Int32 StartingPage, Int32 NumberOfPages) in P:\Cloud\Dropbox\EF Recovery\OCRTest\EFR.OCR\OCR.vb:line 113
I've processed many PDFs through my code, some text, some scans, some mixed together. Some had forms... This is the first time that I've had this error.
Here's a snippet of my code...
Using reader As New iText.Kernel.Pdf.PdfReader(fileInfo.FullName)
reader.SetUnethicalReading(True)
Using sourceDoc As New iText.Kernel.Pdf.PdfDocument(reader)
If NumberOfPages = 0 Then NumberOfPages = sourceDoc.GetNumberOfPages
For i As Integer = StartingPage To StartingPage + NumberOfPages - 1
Dim pageText As String = ""
Try
pageText = iText.Kernel.Pdf.Canvas.Parser.PdfTextExtractor.GetTextFromPage(sourceDoc.GetPage(i))
Catch ex As Exception
OCRLog.Log($"Error attempting to extract text from page {i}. {ex.ToString}")
End Try
If pageText = "" Then
'extract this page
Dim results As OCRResults = ExtractTextFromPDFImagePage(fileInfo.FullName, i)
pageText = results.Text
pageItems.Add(New OCRResults.PagesClass(results.Accuracy, True, pageText))
Else
pageItems.Add(New OCRResults.PagesClass(100, False, pageText))
End If
stringBuilder.Append(pageText)
Next
Return New OCRResults(stringBuilder.ToString, pageItems)
End Using
End Using
Any ideas?
There is an error in the PDF, just as indicated by the error text "'StandardEncoding' is not a supported encoding name.".
The fonts on the page you shared use the name StandardEncoding in their Encoding entries. This is not a valid name here. According to the specification ISO 32000-1 the only valid values here are MacRomanEncoding, MacExpertEncoding, and WinAnsiEncoding, see Table 111 – Entries in a Type 1 font dictionary – and Table 114 – Entries in an encoding dictionary.
Adobe Preflight also complains about these names when checking for syntax errors:
An unexpected value is associated with the key
Key: BaseEncoding
Value: /StandardEncoding
Type: CosName
Formal Representation: Encoding
Cos ID: 38
Traversal Path: ->Pages->Kids->[0]->Resources->Font->WARSP->Encoding
An unexpected value is associated with the key
Key: Encoding
Value: /StandardEncoding
Type: CosName
Formal Representation: Font.FontType1
Cos ID: 27
Traversal Path: ->Pages->Kids->[0]->Resources->Font->Arial,Bold
An unexpected value is associated with the key
Key: BaseEncoding
Value: /StandardEncoding
Type: CosName
Formal Representation: Encoding
Cos ID: 22
Traversal Path: ->Pages->Kids->[0]->Resources->Font->Arial->Encoding
An unexpected value is associated with the key
Key: BaseEncoding
Value: /StandardEncoding
Type: CosName
Formal Representation: Encoding
Cos ID: 19
Traversal Path: ->Pages->Kids->[0]->Resources->Font->ARROW->Encoding
(Excerpt from a preflight report for your shared PDF)
In spite of StandardEncoding not being a valid name here, the PDF specification knows a "Standard Encoding", see Annex D of ISO 32000-1. Most likely your document attempts to refer to that encoding at the locations outlined above.
If you need to extract text from the document in question, therefore, you may want to follow the recommendation of the error message:
For information on defining a custom encoding, see the documentation for the Encoding.RegisterProvider method.
The Encoding class here is the one in System.Text.
To extract the text from your PDF, therefore, it should suffice to implement an EncodingProvider that for the name StandardEncoding provides an Encoding instance according to the information from the STD column of the table in Annex D.2 – Latin Character Set and Encodings – of ISO 32000-1.
I can't call a java wrapping service in Acceleo because it doesn't recognize parameters type. This is my simple test code: the main calls a query stored in Services.mtl, that calls the java service that just return the name of an object "Send"
Main.mtl
[file ('system.P', false, 'UTF-8')]
[for (t : Send | aSystemBehavior.transitions)) ]
[getName(t)/]
[/for]
[/file]
Services.mtl
[query public getName(arg0 : Send) : String
= invoke('myPackage.Services', 'getName(myPackage.Send)', Sequence{arg0})
/]
Services.java
public class Services
{
public String getName(Send t)
{return t.getName();}
}
The Error Log shows:
Invalid result for expression
self.invoke('myPakage.Services',
'getName(myPakage.Send)', Sequence {arg0}) at line 0 in
Module services for query getName(Send). Last recorded value of self
was org.eclipse.emf.ecore.impl.DynamicEObjectImpl#1f00eb36 (eClass:
org.eclipse.emf.ecore.impl.EClassImpl#2c2aade3 (name: Send)
(instanceClassName: null) (abstract: false, interface: false)).
Problem found while generating the file system.P'.
If I use a String as parameter type instead of Send, everything works fine.
Does the package containing the service "Services" has been exported? If not, open the file MANIFEST.MF, go in the runtime tab and add its package to the list of exported packages. Are you sure that your "Send" object has a name? This message only indicates that null was returned by the query getName.
I don't have anymore this problem... I created a new Acceleo project from scratch, and it works. I am not sure what was the problem... maybe it's something about che choice of metamodels to import during the creation of the Module (I have to choose between run-tim and develop-time metamodel).
H2 database with custom function alias defined as:
create alias to_date as $$
java.util.Date toDate(java.lang.String dateString, java.lang.String pattern) {
try {
return new java.text.SimpleDateFormat(javaPattern).parse(dateString);
} catch(java.text.ParseException e) {
throw new java.lang.RuntimeException(e);
}
}
$$;
H2 initialized as:
jdbc:h2:mem:testdb;INIT=runscript from 'classpath:create_alias.sql
This is used in tests, executed for multiple projects concurrently on a Jenkins instance. Sometimes such tests would fail with following error:
Could not get JDBC Connection; nested exception is org.h2.jdbc.JdbcSQLException: Syntax error in SQL statement "javac: file not found: org/h2/dynamic/TO_DATE.java
Usage: javac <options> <source files>
use -help for a list of possible options
"; SQL statement:
create alias to_date as $$
java.util.Date toDate(java.lang.String dateString, java.lang.String pattern) {
....
My guess is that org.h2.util.SourceCompiler is assuming that there is only one instance of h2 running at the time and writes the generated Java source to 'java.io.tmpdir', which is shared among all processes running under same account. I propose following fix:
Index: SourceCompiler.java
===================================================================
--- SourceCompiler.java (revision 5086)
+++ SourceCompiler.java (working copy)
## -40,7 +40,15 ##
*/
final HashMap<String, Class<?>> compiled = New.hashMap();
- private final String compileDir = Utils.getProperty("java.io.tmpdir", ".");
+ private final String compileDir;
+
+ {
+ // use random folder under java.io.tmpdir so multiple h2 could compile at the same time
+ // without overwriting each other files
+ File tmp = File.createTempFile("h2tmp", ".tmp");
+ tmp.mkdir();
+ compileDir = tmp.getAbsolutePath();
+ }
static {
Class<?> clazz;
Should I open the support ticket or there are workarounds for this issue?
You can use javax.tools.JavaCompiler API and provide your own implementation for in-memory JavaFileManager to completely avoid creating those temp files.
BTW, Janino also support javax.tools.JavaCompiler API.
I had the same problem running multiple Jenkins executors and having Arquillian/Wildfly/H2 integration tests configuration. I found a workaround by setting java.io.tmpdir property to the build directory in the test standalone.xml.
I'm trying to control OpenLDAP using Spring LDAPTemplate.
In LDAP, I've groups and users organization units. I'm trying to bind new users into LDAP with group association. (Generic User Account) So when I try to bind a new user, I also put gidNumber attiribute into attributes object. But I'm getting an error like this:
[LDAP: error code 65 - attribute 'gidNumber' not allowed]; nested exception is javax.naming.directory.SchemaViolationException: [LDAP: error code 65 - attribute 'gidNumber' not allowed]; remaining name 'ou=staff'
Here's what I've tried so far:
DistinguishedName dn = new DistinguishedName();
dn.add("ou", "staff");
Attributes attributes = new BasicAttributes();
attributes.put("objectClass", "inetOrgPerson");
attributes.put("uid", username);
attributes.put("givenName", name);
attributes.put("gidNumber", gidNumber.toString());
attributes.put("sn", surname);
attributes.put("cn", name + " " + surname);
attributes.put("userPassword", password);
ldapTemplate.bind(dn, null, attributes);
Here's my schema:
+--> dc=ibu,dc=edu,dc=tr (5)
---> cn=admin
+--> ou=group (1)
| ---> cn=Academic
---> ou=guest
+--> ou=staff (2)
| ---> cn=John Clark
---> ou=student
You have to add another object class called posixAccount. Because the attribute gidNumber belongs to this class. So try adding one more object class as follows:
attributes.put("objectClass", "posixAccount");
An LDAP entry must just have one and only one structural object class. Some broken servers do allow multiple structural object classes, though. Add a structural object class (the one to add depends on the purpose for which the entry is used).