I want to batch-change files which lie in subdirs. The extension of the existing files is *.ada.npp. I would like to change it to *.ada. The thing is there are files already existing with the desired extension *.ada. Those files need to be replaced.
I tried with mmv and mv extensively but was not successfull (see below).
mmv -rd ';*.npp' '\#1'
$ mmv -rd ';*.npp' '\#1'
mechanisation/nav_kernel/acdc_alignment.1.ada.npp , mechanisation/nav_kernel/acdc_types.1.ada.npp , mechanisation/nav_ke
rnel/acdc_types.2.ada.npp , mechanisation/nav_kernel/ahrs_manager.1.ada.npp , mechanisation/nav_kernel/ahrs_manager.2.ad
a.npp , mechanisation/nav_kernel/aim_augmentation.1.ada.npp , mechanisation/nav_kernel/aim_augmentation.2.ada.npp , mech
anisation/nav_kernel/aim_types.1.ada.npp , mechanisation/nav_kernel/aim_types.2.ada.npp , mechanisation/nav_kernel/analy
tic_platform.1.ada.npp , mechanisation/nav_kernel/analytic_platform.2.ada.npp , mechanisation/nav_kernel/earth_model.1.a
da.npp , mechanisation/nav_kernel/earth_model.2.ada.npp , mechanisation/nav_kernel/ecef_conversions.1.ada.npp , mechanis
ation/nav_kernel/ecef_conversions.2.ada.npp , mechanisation/nav_kernel/egm_2008_gravity_model.1.ada.npp , mechanisation/
nav_kernel/egm_2008_gravity_model.2.ada.npp , mechanisation/nav_kernel/fast_strapdown_manager.1.ada.npp , mechanisation/
nav_kernel/fast_strapdown_manager.2.ada.npp , mechanisation/nav_kernel/geoid.1.ada.npp , mechanisation/nav_kernel/geoid.
2.ada.npp , mechanisation/nav_kernel/gps_toolbox.1.ada.npp , mechanisation/nav_kernel/gps_toolbox.2.ada.npp , mechanisat
ion/nav_kernel/gravity_augmentation.1.ada.npp , mechanisation/nav_kernel/gravity_augmentation.2.ada.npp , mechanisation/
nav_kernel/gyrocompass.1.ada.npp , mechanisation/nav_kernel/gyrocompass.2.ada.npp , mechanisation/nav_kernel/heading_irs
_slew.1.ada.npp , mechanisation/nav_kernel/heading_irs_slew.2.ada.npp , mechanisation/nav_kernel/inertial_data_processin
g.1.ada.npp , mechanisation/nav_kernel/inertial_data_processing.2.ada.npp , mechanisation/nav_kernel/inertial_measuremen
t_pipe.1.ada.npp , mechanisation/nav_kernel/inertial_measurement_pipe.2.ada.npp , mechanisation/nav_kernel/input_validat
ion.1.ada.npp , mechanisation/nav_kernel/input_validation.2.ada.npp , mechanisation/nav_kernel/integrity_monitor.1.ada.n
pp , mechanisation/nav_kernel/integrity_monitor.2.ada.npp , mechanisation/nav_kernel/irs_manager.1.ada.npp , mechanisati
on/nav_kernel/irs_manager.2.ada.npp , mechanisation/nav_kernel/magvar.1.ada.npp , mechanisation/nav_kernel/magvar.2.ada.
npp , mechanisation/nav_kernel/main_manager.1.ada.npp , mechanisation/nav_kernel/main_manager.2.ada.npp , mechanisation/
nav_kernel/main_manager_fast.1.ada.npp , mechanisation/nav_kernel/main_manager_fast.2.ada.npp , mechanisation/nav_kernel
/main_manager_slow.1.ada.npp , mechanisation/nav_kernel/main_manager_slow.2.ada.npp , mechanisation/nav_kernel/motion_se
nsor.1.ada.npp , mechanisation/nav_kernel/motion_sensor.2.ada.npp , mechanisation/nav_kernel/navigation.1.ada.npp , mech
anisation/nav_kernel/navigation.2.ada.npp , mechanisation/nav_kernel/output_preparation.1.ada.npp , mechanisation/nav_ke
rnel/output_preparation.2.ada.npp , mechanisation/nav_kernel/output_preparation_fast.1.ada.npp , mechanisation/nav_kerne
l/output_preparation_fast.2.ada.npp , mechanisation/nav_kernel/pressure_altitude_augmentation.1.ada.npp , mechanisation/
nav_kernel/pressure_altitude_augmentation.2.ada.npp , mechanisation/nav_kernel/satellites_bookkeeper.1.ada.npp , mechani
sation/nav_kernel/satellites_bookkeeper.2.ada.npp , mechanisation/nav_kernel/sea_alignment.1.ada.npp , mechanisation/nav
_kernel/sea_alignment.2.ada.npp , mechanisation/nav_kernel/stationary_heading_stabilization.1.ada.npp , mechanisation/na
v_kernel/stationary_heading_stabilization.2.ada.npp , mechanisation/nav_kernel/strapdown_ecef.1.ada.npp , mechanisation/
nav_kernel/strapdown_ecef.2.ada.npp , mechanisation/nav_kernel/true_airspeed_augmentation.1.ada.npp , mechanisation/nav_
kernel/true_airspeed_augmentation.2.ada.npp , mechanisation/nav_kernel/zero_velocity_augmentation.1.ada.npp , mechanisat
ion/nav_kernel/zero_velocity_augmentation.2.ada.npp -> mechanisation/nav_kernel/#1 : collision.
Nothing done.
Use find.
find . -name '*.ada.npp' -exec sh -c '
for f; do
mv -- "$f" "${f%.npp}"
done' _ {} +
This loop will do what you need. Launch it from the top level directory where you want to start your search
IFS="
";for F in `find . -name "*.ada.npp"`;do B1=`echo "$F" | sed 's/.ada.npp$//'`;mv -- "$F" "$B1.ada";done
I have a control file that loads text to oracle but i am trying to strip first 3 digits of Phone number example 1-469-428-5920 to 469-428-5920, please see below control file. what should be the syntax?
I keep getting error below, Please help
thanks
options (ERRORS=1000, SKIP=2)
load data
infile '/clients/vca2/data/API/ADHOC_LOAD/VCA_RP_Hospital_Website_Master_List_VCA.CSV'
replace
into table VCABARK_CDI_DV.RP_HOSPITAL_MASTER_LIST
fields terminated by '|'
OPTIONALLY ENCLOSED BY '"' AND '"'
trailing nullcols
(
HOSPITAL_EXT_NUMBER,
REGION,
HOSPITAL_NAME,
ADDRESS_1,
CITY ,
STATE,
POSTAL_CODE,
PHONE "CASE WHEN SUBSTR(:PHONE,1,2) = '1-' THEN ‘SUBSTR(:PHONE,3)’ ELSE ‘:PHONE’ END",
RCF_PHONE,
FAX ,
EMAIL ,
WEBSITE_CLASS,
DOMAIN,
SHORTNAME,
PMS,
INDICATOR_24_HOUR,
CARECLUB,
HOME_DELIVERY,
APPOINTMENT_CTA,
FREE_FIRST_EXAM,
AAHA ,
AAFP_MEMBER,
CAT_FRIENDLY,
FEAR_FREE ,
LOW_STRESS, E
VECCS,
FACEBOOK_URL,
GMB_URL ,
YELP_URL,
BOOK_URL,
REQUEST_URL,
HOURS_OPERATIONS_URL,
HD_URL_SUBDOMAIN,
HD_URL_ORIGINAL,
PET_PORTAL_URL,
AIS_CLINIC_ID,
AIS_TOKEN
)
You haven't said what error you are getting, but you should have quotes - curly or otherwise - within your case expression. Instead of:
PHONE "CASE WHEN SUBSTR(:PHONE,1,2) = '1-' THEN ‘SUBSTR(:PHONE,3)’ ELSE ‘:PHONE’ END",
use:
PHONE "CASE WHEN SUBSTR(:PHONE,1,2) = '1-' THEN SUBSTR(:PHONE,3) ELSE :PHONE END",
You could also just keep the last 12:
PHONE "SUBSTR(:PHONE, -12, 12)",
I try to send an excel file who was created when i refresh the page. I used the package https://github.com/harishanchu/Laravel-FTP
When i run the code bellow i get an exception like :
pathinfo() expects parameter 1 to be string, array given
I don't want to pick up a file in the storage because the system that i made will run the daily query everyday and send the file via FTP ! so i can't know the path before the file was created.
Hope someone could help me on this ! thanks a lot in advance
here the full code :
$licencies = Licencies::where('lb_assurance' , '=' , 'Lafont')
->leftJoin('activite_licencie' , 'activite_licencie.id' , '=' , 'licencies.activite_licencie_id')
->leftJoin('saisons' , 'saisons.id' , '=' , 'licencies.saison_id')
->leftJoin('pays' , 'pays.id' , '=' , 'licencies.pays_naissance_id')
->leftJoin('type_licence' , 'type_licence.id' , '=' , 'licencies.type_licence_id')
->leftJoin('structures' , 'structures.id' , '=' , 'licencies.structure_id')
->leftJoin('civilite' , 'civilite.id' , '=' , 'licencies.civilite_id')
->leftJoin('catg_licence' , 'catg_licence.id' , '=' , 'licencies.catg_licence_id')
->select('num_licence' , 'civilite.lb_civilite' , 'lb_nom' , 'lb_prenom' , 'dt_naissance' , 'pays.fr' ,'activite_licencie.lb_activite' , 'catg_licence.lb_catg_lic' , 'type_licence.lb_type' ,'saisons.lb_saison', 'lb_surclassement' , 'structures.nom_structure' , 'lb_assurance' , 'cd_dept_naissance' , 'lb_adresse' , 'tel_fix_licencie' , 'tel_port_licencie' , 'adresse_email')
->whereRaw('DATE(licencies.created_at) = CURRENT_DATE')
->get();
$licencies->map(function($licencie) {
$licencie['dt_naissance'] = \Carbon\Carbon::parse($licencie['dt_naissance'])->format('d/m/Y');
//$licencie['created_at'] = \Carbon\Carbon::parse($licencie['created_at'])->format('d/m/Y H:i:s');
return $licencie;
});
$date = Carbon::now('Europe/Paris')->format('d-m-Y H:i:s');
$file = Excel::create('Assurance Lafont - Saisie des Licences FFRXIII le : ' . $date . '' , function($excel) use ($licencies) {
$excel->sheet('Excel', function($sheet) use ($licencies)
{
$sheet->fromArray($licencies);
});
});
$directory = 'mydirectory' ;
$ftp = FTP::connection()->uploadFile($file, $directory);
dd($ftp);
which is the PigStorage() need to use to get the load the file for the following query
INSERT OVERWRITE DIRECTORY 'doop'
select a.* from cdr.cell_tower_info
the output for the above query is like this
Haryana Ambala 404 20 80 37591 76.76746 30.373488 404-20-80-37591
Haryana Ambala 404 20 80 30021 76.76746 30.373488 404-20-80-30021
Haryana Ambala 404 20 80 37591 76.76746 30.373488 404-20-80-37591
I am working with CDR analysis where first i need to retrieve some fields from a table using select and save it into a HDFS folder , this result is again need to load using pig for further analysis
Try this
CREATE EXTERNAL TABLE cell_tower_info
FIELDS TERMINATED BY ','
LOCATION 'doop'
AS
SELECT .* from cdr.cell_tower_info
Hive default delimiter is ctrl-A (\001).
I think PigStorage('\001') should work to use the Hive output data in PIG.
Alternatively Hive table can be defined as fields terminated by '\t' , so that the result can be used directly in PIG.
Can you try this?
input.txt
Haryana Ambala 404 20 80 37591 76.76746 30.373488 404-20-80-37591
Haryana Ambala 404 20 80 30021 76.76746 30.373488 404-20-80-30021
Haryana Ambala 404 20 80 37591 76.76746 30.373488 404-20-80-37591
PigScript:
A = LOAD 'input.txt' AS line;
B = FOREACH A GENERATE FLATTEN(STRSPLIT(line,'\\s+')) AS (f1:chararray,f2:chararray,f3:int,f4:int,f5:int,f6:int,f7:double,f8:double,f9:chararray);
C = FOREACH B GENERATE f1,f2,f8,f9;
DUMP C;
Output:
(Haryana,Ambala,30.373488,404-20-80-37591)
(Haryana,Ambala,30.373488,404-20-80-30021)
(Haryana,Ambala,30.373488,404-20-80-37591)
if your data is stored in hive table then,
X = LOAD 'hdfs://localhost:9000/user/hive/warehouse/input.txt' using PigStorage('\t') AS line;
Y = FOREACH A GENERATE FLATTEN(STRSPLIT(line,'\\s+')) AS (A1:chararray,A2:chararray,A3:int,A4:int,A5:int,A6:int,A7:double,a8:double,A9:chararray);
Z = FOREACH B GENERATE A1,A2,A8,A9;
DUMP Z;
in my case port is 9000. give according to your system.
I need to convert Arabic text from windows-1256 to utf-8 how can I do that? any help?
thanks
Try lua-iconv, which binds iconv to Lua.
local win2utf_list = [[
0x00 0x0000 #NULL
0x01 0x0001 #START OF HEADING
0x02 0x0002 #START OF TEXT
-- Download full text from
-- http://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1256.TXT
0xFD 0x200E #LEFT-TO-RIGHT MARK
0xFE 0x200F #RIGHT-TO-LEFT MARK
0xFF 0x06D2 #ARABIC LETTER YEH BARREE
]]
local win2utf = {}
for w, u in win2utf_list:gmatch'0x(%x%x)%s+0x(%x+)' do
local c, t, h = tonumber(u,16), {}, 128
while c >= h do
t[#t+1] = 128 + c%64
c = math.floor(c/64)
h = h > 32 and 32 or h/2
end
t[#t+1] = 256 - 2*h + c
win2utf[w.char(tonumber(w,16))] =
w.char((table.unpack or unpack)(t)):reverse()
end
local function convert_to_utf8(win_string)
return win_string:gsub('.', win2utf)
end
Windows-1256 is one of the character-sets designed as an 8-bit overlay of ASCII. It therefore has 256 characters, each encoded as one byte.
UTF-8 is an encoding of the Unicode character-set. Being "universal", it works out that it is a superset of the Windows-1256 character-set. So, there is no loss of information by having to use "substitution characters" in place of ones that aren't members of the character-set.
Conversion is a simple matter of transforming the Windows-1256 byte for each character to the corresponding UTF-8 bytes. A lookup table is an easy way to do it.
local encoding = {
-- table maps the one byte Windows-1256 encoding for a character to a Lua string with the UTF-8 encoding for the character
"\000" , "\001" , "\002" , "\003" , "\004" , "\005" , "\006" , "\007" ,
"\008" , "\009" , "\010" , "\011" , "\012" , "\013" , "\014" , "\015" ,
"\016" , "\017" , "\018" , "\019" , "\020" , "\021" , "\022" , "\023" ,
"\024" , "\025" , "\026" , "\027" , "\028" , "\029" , "\030" , "\031" ,
"\032" , "\033" , "\034" , "\035" , "\036" , "\037" , "\038" , "\039" ,
"\040" , "\041" , "\042" , "\043" , "\044" , "\045" , "\046" , "\047" ,
"\048" , "\049" , "\050" , "\051" , "\052" , "\053" , "\054" , "\055" ,
"\056" , "\057" , "\058" , "\059" , "\060" , "\061" , "\062" , "\063" ,
"\064" , "\065" , "\066" , "\067" , "\068" , "\069" , "\070" , "\071" ,
"\072" , "\073" , "\074" , "\075" , "\076" , "\077" , "\078" , "\079" ,
"\080" , "\081" , "\082" , "\083" , "\084" , "\085" , "\086" , "\087" ,
"\088" , "\089" , "\090" , "\091" , "\092" , "\093" , "\094" , "\095" ,
"\096" , "\097" , "\098" , "\099" , "\100" , "\101" , "\102" , "\103" ,
"\104" , "\105" , "\106" , "\107" , "\108" , "\109" , "\110" , "\111" ,
"\112" , "\113" , "\114" , "\115" , "\116" , "\117" , "\118" , "\119" ,
"\120" , "\121" , "\122" , "\123" , "\124" , "\125" , "\126" , "\127" ,
"\226\130\172", "\217\190" , "\226\128\154", "\198\146" , "\226\128\158", "\226\128\166", "\226\128\160", "\226\128\161",
"\203\134" , "\226\128\176", "\217\185" , "\226\128\185", "\197\146" , "\218\134" , "\218\152" , "\218\136" ,
"\218\175" , "\226\128\152", "\226\128\153", "\226\128\156", "\226\128\157", "\226\128\162", "\226\128\147", "\226\128\148",
"\218\169" , "\226\132\162", "\218\145" , "\226\128\186", "\197\147" , "\226\128\140", "\226\128\141", "\218\186" ,
"\194\160" , "\216\140" , "\194\162" , "\194\163" , "\194\164" , "\194\165" , "\194\166" , "\194\167" ,
"\194\168" , "\194\169" , "\218\190" , "\194\171" , "\194\172" , "\194\173" , "\194\174" , "\194\175" ,
"\194\176" , "\194\177" , "\194\178" , "\194\179" , "\194\180" , "\194\181" , "\194\182" , "\194\183" ,
"\194\184" , "\194\185" , "\216\155" , "\194\187" , "\194\188" , "\194\189" , "\194\190" , "\216\159" ,
"\219\129" , "\216\161" , "\216\162" , "\216\163" , "\216\164" , "\216\165" , "\216\166" , "\216\167" ,
"\216\168" , "\216\169" , "\216\170" , "\216\171" , "\216\172" , "\216\173" , "\216\174" , "\216\175" ,
"\216\176" , "\216\177" , "\216\178" , "\216\179" , "\216\180" , "\216\181" , "\216\182" , "\195\151" ,
"\216\183" , "\216\184" , "\216\185" , "\216\186" , "\217\128" , "\217\129" , "\217\130" , "\217\131" ,
"\195\160" , "\217\132" , "\195\162" , "\217\133" , "\217\134" , "\217\135" , "\217\136" , "\195\167" ,
"\195\168" , "\195\169" , "\195\170" , "\195\171" , "\217\137" , "\217\138" , "\195\174" , "\195\175" ,
"\217\139" , "\217\140" , "\217\141" , "\217\142" , "\195\180" , "\217\143" , "\217\144" , "\195\183" ,
"\217\145" , "\195\185" , "\217\146" , "\195\187" , "\195\188" , "\226\128\142", "\226\128\143", "\219\146"
}
--
encoding.convert = function(str)
assert(type(str) == "string", "Parameter 1 must be a string")
local result = {}
for i = 1, string.len(str) do
table.insert(result, encoding[string.byte(str,i)+1])
end
return table.concat(result)
end
assert(encoding.convert("test1") == "test1", "test1 failed")
Refs:
Joel Spolsky, The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)
Roberto Ierusalimschy, Creating Strings Piece by Piece
Generally, convert from a code page (character set) to another. Must use a map table.
Which like: http://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1256.TXT, from CP1256 to Unicode.
Then convert from Unicode to Utf8 (encode/decode method works between Unicode & UTF-8 , without a big map).