I have a source file that contains two fields: IP_ADDRESS and USER_NAME. I want to check whether the IP address is correct or not before loading it to the datawarehouse using DATASTAGE. How do I do this?
I was browsing Stack Overflow and think I might have a solution to your question. If you create a job to grab all of the IP_ADDRESS's from the file and send them to a BASIC transformer (search for BASIC transformer in DataStage. It is NOT the one that is normally on the palette). From there, set the Stage Variables as 'SetUserStatus() and write out the column name to a peek stage (You don't need the output at all. The SetUserStatus is the important part). This will now allow you to pass up the Command Output (list of IP Addresses) to a Sequence. From the Sequence, start with the job you just created (BASIC transformer job) and link that to a User Variables Activity. In the User Variables Activity stage, Set the name to something like 'IP Address' and Expression as IP_ADDRESS.$UserStatus. You can then use a Loop to take that output that is now a List and send each individual IP Address to an Execute Stage with a Ping command to see if it returns a valid IP Address. If it does return a valid IP then have your job that writes the USER_NAME and IP_ADDRESS to do a 'Select' statement where the IP_ADDRESS = the valid IP_ADDRESS. For the ones that aren't valid, you can send them down a different path and have them write out to '.txt' file somewhere so you know which ones weren't valid. I'm sure you will need a few more steps in there but that should be the gist of it.
Hope my quick stab at your issue helps.
Yes, you can use a transformer or a transformer and a filter to do that, depending on the version of Datastage you're using. If you're using PX, just encode the validation logic in a transformer stage, and then, on the output link set up a filter that doesn't allow the rows to pass forward if they didn't pass the validation logic.
Related
Team,
Occasionally my flow fails and its enough test it manually to running again. However, I want to avoid that this error ocurrs again to stay in calm.
The error that appears is this:
Unable to process template language expressions in action 'Periodo' inputs at line '0' and column '0': 'The template language function 'split' expects its first parameter to be of type string. The provided value is of type 'Null'. Please see https://aka.ms/logicexpressions#split for usage details.'.
And it appears in 2 of the 4 variables that I create:
Client and Periodo
The variable Clientlooks this:
The same scenario to "Periodo".
The variables are build in the same way:
His formula:
trim(first(split(first(skip(split(outputs('Compos'),'client = '),1)),'indicator')))
His formula:
trim(first(split(first(skip(split(outputs('Compos'),'period = '),1)),'DATA_REPORT_DELIVERY')))
The same scenario to the 4 variables. 4 of them strings (numbers).
Also I attached email example where I extract the info:
CO NIV ICE REFRESCOS DE SOYA has finished successfully.CO NIV ICE REFRESCOS DE SOYA
User
binary.struggle#mail.com
Parameters
output = 7
country = 170
period = 202204012
DATA_REPORT_DELIVERY = NO
read_persistance = YES
write_persistance = YES
client = 18277
indicator_group = SALES
Could you give some help? I reach some attepmpts succeded but it fails for no apparent reason:
Thank you.
I'm not sure if you're interested but I'd do it a slightly different way. It's a little more verbose but it will work and it makes your expressions a lot simpler.
I've just taken two of your desired outputs and provided a solution for those, one being client and the other being country. You can apply the other two as need be given it's the same pattern.
If I take client for example, this is the concept.
Initialize Data
This is your string that you provided in your question.
Initialize Split Lines
This will split up your string for each new line. The expression for this step is ...
split(variables('Data'), '\n')
However, you can't just enter that expression into the editor, you need to do it and then edit in in code view and change it from \\n to \n.
Filter For 'client'
This will filter the array created from the split line step and find the item that contains the word client.
`contains(item(), 'client')`
On the other parallel branches, you'd change out the word to whatever you're searching for, e.g. country.
This should give us a single item array with a string.
Initialize 'client'
Finally, we want to extract the value on the right hand side of the equals sign. The expression for this is ...
trim(split(body('Filter_For_''client''')[0], '=')[1])
Again, just change out the body name for the other action in each case.
I need to put body('Filter_For_''client''')[0] and specify the first item in an array because the filter step returns an array. We're going to assume the length is always 1.
Result
You can see from all of that, you have the value as need be. Like I said, it's a little more verbose but (I think) easier to follow and troubleshoot if something goes wrong.
I have this Analysis that I wish to invoke from URL (which I can).
This analysis in particular is mounted on a dashboard and 6 of the 17 columns can be filtered from a prompt mounted on the same dashboard.
I only need the analysis and still be able to filter the results. I read about parameters that can be given to the URL, but I can't seem to make the filters work. I am using the &P0=n parameter and the consequent &P1=xx parameter.
Without the &P I am able to see the table, when I add parameters I get this error UH6MBRBC:E6MUPJPH.
Thanks for your time, have a nice day.
Got it to work by myself afterall (always best).
Basically instead of &p0 and such I used &coln and &valn (where n is the column number):
&coln - is the name of the field
&valn - is the value for which you filter
More information and better explanation can be found here.
When I run the following command from Mikrotik SSH, I see an asterisk followed by a hex value, which will look something like like *4e.
:put [:execute { :ping localhost count=10 }]
According to the Mikrotik wiki, the :execute command executes commands in the background as a script job. When I print out the currently running jobs, I don't see any relation between the output above with the output of this.
[admin#MikroTik] > /system script job print
# SCRIPT OWNER STARTED
0 admin jun/05/2017 16:58:09
1 admin jun/05/2017 17:07:31
So, what does the output represent and how can I use it?
The return that you get is a MikroTik RouterOS Internal ID. In this case, this is an internal ID for the running job. Internal IDs are immutable references to the object which they represent and have the advantage of being the same between command executions even if concurrent operations are taking place which would change the objects # as shown by print.
You can identify an internal ID by the * followed by a hexadecimal number. In some cases, where there are multiple items to return as is common with find, you will get a ; delimited list of IDs such as:
*d;*1;*18;*3;*19;*1a;*20
Internal IDs can be used as when running other commands just like the printable number and is distinguished by the preceding *. Where a list of multiple internal ids is given, the command will run on each item.
Information on the internal ID data type can be found in the Scripting Manual. Internal IDs do not follow any ordering and you should not assume that they do. To get the internal ID of an object from its ID show in a print output, you can use :put [get <id>] where <id> is the printable id - the output will show the internal id as the .id property.
Using TeamCity 9.1.4.
I'm trying to get some server hostnames into a Command Line script with Configuration Parameters. I want each option to contain multiple hostnames.
My configuration:
vanmain => rad-ecr1,rad-ecr2,rad-ecr3,rad-myecr,rad-balancer
tor => rad2-bal,rad2-ecr1,rad2-ecr2,rad2-myecr
fvcdc => rad-fvcdc,rad-balancer
bccfa => rad-bccfa
When I select fvcdc in a build, I receive the following error message:
One of entered values 'rad-fvcdc' is not one of valid select item values: rad-ecr1,rad-ecr2,rad-ecr3,rad-myecr,rad-balancer,rad2-bal,rad2-ecr1,rad2-ecr2,rad2-myecr,rad-fvcdc,rad-balancer,rad-bccfa
How do I get the values into my script?
Dunkan,
I successfully reproduced your issue and was able to find out the root cause of it.
On my virtual installation I created a build with select type parameter, let's name it HostValue. Next, in Items field I copy/pasted values from your initial post and tried to reproduce the problem -- but executing the build was successful. Then I decided to reconfigure parameter and toggled Allow multiple checkbox, and viola, same error message as you got!
If you read small text below Value separator field, you will see, that default value is comma: , and as your values contain this symbol you got an error.
So, to solve this problem I can suggest you these variants:
If you don't need multiple choices, you can just turn off this feature and everything should work.
Replace default Value separator with custom one, for example <SEP>. Then whenever you will select multiple values for this parameter you will get something like:
"rad-ecr1,rad-ecr2,rad-ecr3,rad-myecr,rad-balancer"<SEP>"rad2-bal,rad2-ecr1,rad2-ecr2,rad2-myecr"<SEP>"rad-fvcdc,rad-balancer"
Replace comma in your values with some other separator, for example | or :. In this case it would look like:
"rad-ecr1:rad-ecr2:rad-ecr3:rad-myecr:rad-balancer","rad-fvcdc:rad-balancer"
After that you can use the value of this parameter as usual %HostValue% and parse depending on which variant you choose.
Maybe the error message from server could be a little bit clearer. Hope it will help you.
Also I would like to recommend you my plugin teamcity-web-parameters. It will allow you to create dynamic select values from external web service.
Take a look at this thread: https://teamcity-support.jetbrains.com/hc/en-us/community/posts/206843785-How-to-specify-empty-value-for-Typed-Parameter -- looks very similar to your question.
I am assigned to implement new custom fields. The possible entries have to be selected from a value table. The value table holds the key and a explaning text. The dynpro is build like that:
Input for new field (zzfield) | Output-only field for explaining text. (zzfield_text)
What I want my dynpro to do:
a) Show text in output field based on entered key
b) Update text after ENTER/SAVE from user
c) Update text directly after a new key has been selected from the value table
My current solution is this:
" in the dynpro
PROCESS BEFORE OUTPUT.
MODULE GET_ZZFIELD_TEXT.
" in the module
DATA: zzfield_text TYPE string.
SELECT SINGLE text
FROM value_table
INTO zzfield_text
WHERE zzfield = mara-zzfield.
This works well for the cases a) and b) I have listed above. However, beeing in a PBO, it does not work for case c). My colleage suggested to use something like this:
" in the dynpro
PROCESS AFTER INPUT.
field: zzfield MODULE get_zzfield_text_pai ON REQUEST.
I can not duplicate my code in the module get_zzfield_text_pai, because the of the DATA declaration of zzfield_text. It is reported to be unknown in the PAI (besides it is already defined in the PBO module) but it is also reported to be duplicated if I declare it again in the PAI. Further, I dont know how to implement new global fields into my function group. Using another include creates again issues of that zzfield_text beeing unknown.
In general, I am not happy about my solution and would be very thankfull for each suggestion about how to code this feature.