SGP.02 - Remote Provisioning Architecture for Embedded UICC Technical Specification (page 255 of v4.0 specifically) says:
the data format provided by the function caller SHALL NOT depend of the selected OTA protocol capabilities (for example SM-DP can consider there is no limit on data length)
and later
The SM-SR has the responsibility to build the final Command script,
depending on eUICC capabilities and selected protocol:
by adding the Command scripting template for definite or indefinite length,
and, if necessary, by segmenting the provided command script into several pieces
and, if necessary, by adding the relevant Script Chaining TLVs.
I understand it that SM-DP can send arbitrary long data parameter to ES3.SendData and SM-SR should send several APDUs in multiple SMSes if data is too large to fit into one. And that is meant by segmenting.
The problem is I can't find a relevant specification that defines how segmenting should be done. And that's the question: where is the segmenting process defined?
I may be wrong but it seems that is not the same as Concatenated Short Messages described in the section 6.3 of ETSI TS 123 048.
It seems that Script Chaining briefly mentioned in ETSI TS 102 226 is somewhat related so pointers to the specification that defines how it works are also very much welcome (TS 102 226 talks about Script Chaining TLVs but not how to use them, at least I'm definitely missing some broader context how it works so any hints are appreciated).
UPDATE:
ES8.EstablishISDPKeySet function requires 3 APDUs to be sent. And they are quite big as they contain keys. From SGP.02-v4.0 table 150 I understand it that they are sent from SM-DP to SM-SR using Expanded Remote Command Format. The script in this format can be rather large as far as I understand (given that SM-DP can assume that there is no limitation on the data length). And it is not clear how SM-SR should segment it or use chaining. I'm just missing the specs where this is described.
The eUICC has a limited internal buffer, i.e. it cannot store a complete profile package of 10kb or more internally. The message must be chunked. If the eUICC supports just e.g. 1 kb then you have to split the APDU commands after at most 3 APDU commands to stay below 1kb. The SGP.02 specification defines to have at least 1024 bytes. A fully featured SM-SR might store some attributes based on the eUICC vendor in the EID to add special handling and patches for certain eUICCs to support a larger buffer size.
Encode each APDUs chunk (1..n APDUs) into the Expanded Remote Command Format (ETSI TS 102 226, sect. 5.2.1) (compact format can only have one response for the last APDU, but if it works, you could save a few bytes)
Encode each Expanded Remote Commands message into a SMS-DELIVER (TS 123 048 and simalliance Interoperability Stepping Stones release 6) This includes the data encryption using the OTA keys (KiC, KID). gsm0348 is a good Java library for this. Pay attention: For each message the OTA counter must be incremented. Select one reference number and keep it for all messages. I guess this is the identifier how the eUICC knows which messages belong together.
If using SMS (I would suggest to use CAT-TP or HTTPs instead -> faster and more reliable) encode it as SMS-PP Download message (TS 131 111). You will using message concatenation here if the payload is more than 140 bytes.
You will receive as response a SendShortMessage (TS 131 111, 6.4.10). Extract the user data again with TS 123 048 and simalliance Interoperability Stepping Stones release 6. You get the SMS Response message. Look into the response packet to get the user data.
Extract the user data as Expanded Remote Response (ETSI TS 102 226)
The eUICC will handle the streamed messages. The concatenated short messages are only used for a message chunk belonging together during the transit.
The best specification that kind of explains how segmenting and script chaining works in details is SGP.11 Remote Provisioning Architecture for Embedded UICC Test Specification.
It does not have a requirements per se but it has the byte level examples of incoming ES3.SendData and examples of the messages received by eUICC. And this allows to deduce the actual behaviour of the SM-SR pretty easily.
And here is more detailed explanation with references to that specification.
Command script
Command script is a list of commands sent in the data fields of the ES3.SendData. It can be (see data field description in table 150 of SGP.02-v4.0) either:
the list of C-APDUs commands (defined in EXPANDED_COMMANDS method on page 595 of SGP.11-v4.0 and uses in step 2 on page 407 of SGP.11-v4.0)
TLV commands that form a SCP03t script (defined in SCP03T_SCRIPT method on page 600 of SGP.11-v4.0 and used in step 14 on page 408 of SGP.11-v4.0)
Script Chaining
Script chaining is the feature used when there is a need to execute several commands sent in multiple transport messages but when some context of the execution needs to preserved after the first command is executed and used for the next one (for example the first command is INSTALL [for personalization] and the second one is STORE DATA). This context is command session defined in section 4 of TS 102 226.
This is used at least in these cases (which is also supported by Annex K in SGP.02):
when the followup command is based on the response from eUICC
when command script is too large and cannot be sent in one piece
Script chaining is performed by SM-SR and is implemented by adding Script Chaining TLV to the command script in Expanded Remote format. See section 5.4.1.2 in TS 102 226.
Examples of script chaining can be found in SGP.11-v4.0.
For the first case see the scenario for EstablishISDRKeyset procedure in section 4.2.10.2.1.1 . For the second case see the scenario for DownloadAndInstallation procedure in section 4.2.18.2.1.1. The byte level representation of the script chaining is described in SCP80_PACKET method on page 601 (see option CHAINING_OPT).
It seems that explicit chaining is applicable only to SMS or CAT_TP transport. For HTTPS the command session coincides with the administrative session as defined in section 3.6 of Remote Application Management over HTTP – Public Release v1.1.3 so there is not need to add explicit Script Chaining TLVs.
Segmenting
When the command script is too large and cannot be sent in one piece SMSR generates multiple command scripts. The commands that SMSR received from SMDP CMD1, CMD2, …, CMDN (see step 14 on page 408 of SGP.11-v4.0) and that form a command script are split into several command scripts. The first one contains some initial portion of the commands CMD1, CMD2, …, CMDi. The second command script contains commands CMDi+1, CMDi+2, …, CMDj. The third one CMDj+1,…,CMDk and so on.
If SMS or CAT_TP is used then for every command a Chaining TLV is added to the beginning with the appropriate value (see section 4.2.18.2.1.1 in SGP.11-v4.0).
And then every new command script is sent in a separate message to eUICC.
I'm using SysInternal's handle.exe and I'm trying to understand the output.
Here's a snippet:
24C: File (RW-) C:\Program Files (x86)\Google\Chrome\Application\Dictionaries\en-US-8-0.bdic
2E8: Section \Sessions\1\BaseNamedObjects\CrSharedMem_5ae414b12a307dbddc3f42b8b35edcbf313107945050b3aaab1602ecd937c940
2F4: Section \Sessions\1\BaseNamedObjects\CrSharedMem_ccfa88ab65617b75dbdcb72cb6512bf1a9cc76d07a25e9f770b46f4f7c2234bf
314: File (R--) C:\Windows\Fonts\arial.ttf
324: File (R--) C:\Windows\Fonts\arialbd.ttf
328: File (R--) C:\Windows\Fonts\arialbi.ttf
What does the number at the start mean?
What does "Section" mean? I can understand an open file, but what's an open section?
What does the RWD triplet mean? I'm guessing R and W are read and write, but what's D?
The first column is the HANDLE value, it serves as the unique identifier of the OS kernel object. Like the ID column of a database record. It is only useful if you need to compare it with what the debugger tells you when you debug code.
The second column identifies the type of OS object. "File" is obvious, a "Section" is an object that allows processes to share memory. "Memory mapped file" is the usual phrase in programming. "Mutant" tends to be confusing, it is a mutex in normal speech. The author of the program uses the kind of terms that David Cutler likes, he speaks with a VMS lisp. The WinObj utility is another way to look at these kernel objects.
The letters in parentheses are the sharing options specified when the object was created. The third argument to CreateFile. Important to know since it tells you what another program can do when it also wants to access the object. R says that it can read, W says that it can write, D says that it can delete the object without affecting anybody else that uses the object. The object won't be destroyed until everybody closes their handle. An anti-malware scanner or search indexer are typical examples of programs that use delete sharing.
Is there a way to create a console application program that asks input data (eg users birth dates, favourite food, everything) and does anything i program it to do and keep those data stored in that .exe when i close it?
This way, when I'll open it again, all those datas will still be saved there, so I just have to update or modify them.
Don't save data inside your executable but outside of it
Is there a way to create a [...] program that asks input data [...] then keep those data stored in that .exe when i close it?
There is no simple way (but you don't need that). What you want is related to persistence and application checkpointing.
In practice, you probably want to store data in a file -not your executable- (perhaps using some textual format like JSON) or in some database (perhaps as simple as some sqlite, or interacting with some RDBMS like PostGreSQL). For things like birthdays and food preference, an sqlite database file is probably the good approach (see some SQLite tutorial). Put efforts in the good design of your database schema.
This way, when I'll open it again, all those datas will still be saved there
Those datas will still be there if you keep them in some outside file (perhaps a simple myappdata.sqlite one). You can easily design your program to create that file if it does not exist (this happens only the first time you run your program; on the next runs, your program would successfully read that data from that outside file at startup).
In most current operating systems (read this textbook to learn more about OSes) notably Windows, MacOSX, Linux, Android, ..., the executable is supposed to be read-only. And it might be running in several processes at the same time (in such case, what should happen? Think of ACID properties).
The usual practice is to store data outside of the executable (most programs, including your text processor, your compiler, your web browser, ... are doing that). You don't explain why you want to store some data inside the executable, and doing so is unusual and highly operating system specific and executable format specific (for Linux, study carefully elf(5)...)
I would suggest to save the data in some optional file (or database) - its filepath could have some wired-in constant default, etc.... At startup, you check the existence of that data (e.g. with access(2) on POSIX, or just by handling the failure case of fopen or sqlite3_open etc....). If it does not exist, you initialize your program data somehow. At exit (or save time), you write that data. BTW most programs are doing so.
Notice that on most operating systems and computers, a software is not simply a single executable file, but much more (e.g. required libraries and dependencies, configuration files, data files, build automation scripting such as Makefile, etc...). Its installation is a well identified technical process (sometimes a quite complex one), and package managers are helpful.
My feeling is that without specific motivation, you should not even try to store (mutable) data (persistently) in your executable (it is complex, brittle since very OS & compiler and build-chain specific, unusual, and opens vulnerabilities).
For completeness, some programs did actually write some data by rewriting their executable. On Linux, GNU emacs is doing that (in practice, only during its installation procedure) in its unexec.c file (very brittle, since OS & compiler specific) but that feature is disputed and is likely to disappear.
Many other systems deal cleverly with orthogonal persistence: SBCL has some save-lisp-and-die primitive (it usually persists the state in some other "image" file). Poly/ML has some export facility. J.Pitrat's CAIA system (see this paper and his blog; a 2016 tarball of CAIA is available -with permission- on my home page) is able to regenerate entirely all its C code and all the required data (in thousands of files). FullPliant is persisting its state in a well organized file tree. Such persistence or checkpointing techniques are tied to garbage collection (so you should then read the GC handbook) and are using techniques and algorithms close to copying garbage collection.
FWIW, my current project, bismon, is orthogonally persisting its entire heap, but do that outside of the main executable (in an ideal world, I would like to re-generate all the C or C++ source code of it; I am far from that goal).
My recommendation is to keep your software in several files: its executable, the C++ source code related to it, its data files (and probably much more dependencies, i.e. shared libraries or DLLs, font and image files, required binaries, etc...). Then you don't need to overwrite your executable when persisting your state. Since you mention C++ (which is not homoiconic), you could generate the C++ code of your system (then called a Quine program) with its persistent data (and leave the recompilation of all that generated C++ to the system's C++ compiler). I also recommend to make your self-generating program some free software. (if you do that, be nice to edit your question to gives its URL).
In C++, you might keep the data inside the executable (again, it is a bad idea, and I hope to have convinced you to avoid that approach) in the following way: You add one C or C++ source file (e.g. mydata.cc) which contains only data (e.g. some big const char data[]="... many lines of data ...";) - BTW, the XBM file format could be inspirational. You keep all the other *.o object files (in a place known to your program). To save data, you regenerate that mydata.cc file (with the new data for your current state) at each save operation, and at last you run the appropriate commands (perhaps using std::system in your code) to compile that mydata.cc and link it with the kept *.o into a fresh executable. So every save operation requires the recompilation of data.cc and its linking with other *.o object files (and of course the C++ compiler and linker, perhaps with additional build automation tools, becomes a required dependency of your program). Such an approach is not simpler than keeping an external data file (and requires to keep those *.o object files anyway).
This way, when I'll open it again, all those datas will still be saved there
If your goal is just to get the data if it was written in the past, just keep the data in some optional database or file (as many programs do: your word processor would ask you to save its document before exiting if you start it without any document and write a few words in it) outside of your executable and write it before exiting your program. No need to overwrite your executable!
What you want to do requires the ability to write into (and possibly read from) an executable while it is running. As far as I know this is not possible.
While it is possible to change the behaviour of a running executable based on the user input which it is pre-conditioned to receive (think of a video game), it is not possible to store those inputs directly into the exe.
Video games store the progress, points of the player (which are the result of the inputs from the player) into a file(s) outside the running .exe.
So you will have to store the data in a file outside of the .exe file.
I normally use google protocol buffers to do this.
A good explanation of them can be found here.
They are free, simple to use and supported for C++.
They are better than other formats like XML.
Some of the advantages are mentioned here
Protocol buffers have many advantages over XML for serializing structured data.
Protocol buffers:.
are simpler
are 3 to 10 times smaller
are 20 to 100 times faster
are less ambiguous
generate data access classes that are easier to use programmatically
managing birth dates
As I explain in my other answer (which you should read before this one), you don't want to save data inside your .exe file.
But I am guessing that you want to keep user birth date (and other data) from one run to the next. This answer focus mostly on that "users birth date" aspect and guesses that your question is some XY problem (you really care about birth dates, not about overwriting an executable).
So, you decide to keep them somewhere (but outside of your executable). That could be a textual file; perhaps using JSON or YAML format, or some other textual file format that you define properly, specified in EBNF notation in some document; or a binary file (perhaps protocol buffers as suggested by P.W, or some sqlite "database file", or your own binary format that you need to document properly). It is very important to document properly the file format you are using.
Dealing with a textual file (whose format you have well defined) is easy with just fopen. You first need to define well defined a file path, perhaps as simple as
#define MYAPP_DATA_PATH "mydata.txt"
or better
const char* myapp_data_path = "mydata.txt";
(in reality, you better use some absolute file path to be able to run your program from various working directories, and provide some way to redefine it, e.g. thru program options, i.e. command-line arguments)
You might also need to organize some data structure (a global variable MyData global_data; perhaps) keeping that data. In C++, you'll define some class MyData; and you want it to have at least member functions like void MyData::add_birth_date(const std::string& person, const std::chrono::time_point& birthdate); and void MyData::remove_birth_date(const std::string& person);. Probably you'll have more classes like class Person; etc...
using textual format
So your application starts first by filling global_data if a file mydata.txt exists (otherwise, your global_data keeps its empty initial state). That is simple, you'll have some initialization function like:
void initial_fill_global_data(void) {
std::ifstream input(myapp_data_path);
// the file opening could have failed.... then we return immediately
if (!input || !input.good() || input.fail())
return;
Of course, you need to parse that input. Use well known parsing techniques that would call global_data.add_birth_date appropriately. Notice that for the JSON format, you'll find good C++ libraries (such as jsoncpp) to make that really easy.
Before exiting your application, you should save that file. So you would call a save_global_data function wich outputs into the mydata.txt file the contents of MyData. BTW, you could even register it with std::atexit.
The functions initial_fill_global_data and save_global_data could be member functions (perhaps static ones) of your MyData class.
You might want your program to lock the data file. so that two processes running your program won't make havoc. This is operating system specific (e.g. flock(2) on Linux).
using an sqlite database file
I also suggested to keep your data in an sqlite database file. Read some sqlite tutorial and refer to sqlite C & C++ interface reference documentation. Then you need to think of a well designed database schema. And you don't need anymore to keep all the data in memory, since sqlite is capable of managing a big amount of data (many gigabytes), more that what fits in memory.
Obviously you need a global database pointer. So declare some global sqlite3*db;. Of course, myapp_data_path is now some "mydata.sqlite" path. Your main starts by opening that (and creating an empty database if necessary) using
int opsta = sqlite3_open(myapp_data_path, &db);
if (opsta != SQLITE_OK) {
std::cerr << "failed to open database " << myapp_data_path
<< " with error#" << opsta << "=" << sqlite_errstr(opsta)
<< std::endl;
exit (EXIT_FAILURE);
}
If the database did not exist, it is created empty. In that case, you need to define appropriate tables and indexes in it. My first suggestion could be something as simple as
char* errormsg = NULL;
int errcod = sqlite3_exec(db,
"CREATE TABLE IF NOT EXISTS data_table ("
" name STRING NOT NULL UNIQUE,"
" birthdate INT"
")",
&errormsg);
if (errcod != SQLITE_OK) {
std::cerr << "failed to create data_table " << errormsg << std::endl;
exit(EXIT_FAILURE);
}
Of course, you need to think of some more clever database schema (in reality you want several tables, some database normalization, and you should cleverly add indexes on your tables), and to prepare the queries (transform them into sqlite_stmt-s) done in your program.
Obviously you should not save data inside your executable. In all my approaches above, your myapp program behaves as you want it to be. The first time it is running, it initialize some data -outside of the myapp executable- on the disk if that data was missing. The next times, it reuses and updates that data. But that myapp executable is never rewritten when running.
I want to send "call_language" variable from main IVR to second one. I am using blind transfer to connect this two IVR's. For both of them I have two different VDN's. Right now, the connection is going as illustrated in a picture. After transfer, main IVR goes to "SM"(Session Manager) and then to "CM"(Communicatin Manager). In "CM" it looks for VDN and then goes back to "Experience Portal" through "SM", where it finds the corresponding VDN, and transfers to second IVR.
1)Is there a way to send a variable from main to second IVR?
2)Is there a way to connect these two IVR's directly, without going back to SM and CM?
You stated the requirements as passing the chose language but not the whole context, so I give you my wild guesses:
You can have a key-value pair store. The first script sets the
chosen language as value, the key is the UCID. The second script
reads the value from the store with the key UCID. Since the call
does not leave the CM the UCID should be the same for the whole
time.
If the goal is to have the second script to speak the language the caller chose in the first one then you can use several VDN-s for it. For example the use chose 1 [en] then you transfer the call to VDN 2001. If the use chose 2 [de] then you transfer to 2002. For both VDNs the same script starts but it can decide which language to use.
You can start the second script from the first directly but you need to use hooks and some hacking in the source code so this is the most complex and needs the knowledge.
Could somebody explain I/O to me? From everything I'm gathering, it can be summed up, abstractly, as the way computers interact with humans and vice versa. The I/O channel, or the "how", can run the gamut depending on external devices and/or internal OS management.
So what does the IO class in Ruby do? And how is it different from that of Java or C?
And take this code for instance:
x = IO.sysopen("file_name")
p x
The return is a Fixnum based on the file descriptor. In this case, the "file_name" is a pdf file and return a 7. What does the return object mean?
First of all, sysopen is a very low-level way of interacting with the system. For normal input and output in Ruby, you should use File.open instead.
The number returned by sysopen is called a "file descriptor". It's essentially an index into an array, but not a Ruby array; it lives inside the part of a process's memory which is maintained by the operating system. The first file descriptor, number 0, is called "standard input". Input calls will read from this input stream by default. The second, 1, is called "standard output"; output calls send their output there by default. And the third, 2, is called "standard error", which is where error messages go. All three of those are opened by the operating system before Ruby even starts. Normally they're all tied to the terminal, but you can change that with shell redirection.
As a general rule, when you open an extra file, the first one you open will get file descriptor 3, the next 4, and so on. So if you get a 7 back, that just means that Ruby has opened 4 other files by the time it gets to your code. And that's all it means. You can't tell anything else about an open file just based on the number. You have to hand that number off to a system call which can go look at the file descriptor array to see what's up.
But in Ruby, you usually have no reason to know or care about file descriptor numbers. You deal with instances of the IO class (and its subclasses like File for specific types of I/O). You call methods on the IO objects, and they handle the details of the system calls for you. The object referred to by the predefined constant STDIN (which is also the initial value of the global variable $stdin) knows that its file descriptor is 0, so you don't have to know that.