Related
This question is about converting a gzip deflate message from a websocket message and convert it to array OR raw text that I can apply JSON.parse on it...
*** to be also clear: In this question : i use a websocket from a crypto exchange.... but the question is about the received message NOT about crypto exchange
in the documentation they say "please use zlib deflate"
HERE THE JAVASCRIPT
digifinexopen = '{"id":12312,"method":"trades.subscribe","params":["btc_usdt"]}';
digifinex_market_ws = new WebSocket("wss://openapi.digifinex.com/ws/v1/");
digifinex_market_ws.binaryType = "arraybuffer";
digifinex_market_ws.onmessage = event => digifinex_trades(event.data);
digifinex_market_ws.onopen = event => digifinex_market_ws.send(digifinexopen);
function fu_bitmex_trades (jsonx) { console.log(jsonx); }
I have this in the log
object=>[[Int8Array]]: Int8Array(1129) 0 … 99]
0: 120
1: -38
I tried with <script src="https://cdnjs.cloudflare.com/ajax/libs/pako/2.0.4/pako.min.js" ...></script>
if I do pako.deflate(jsonx);
I get
object=> Uint8Array(78) [120, 156, 1, 67, 0, 188, 255, 120, 218, 4, 192, 177, 13, 196, 32, 12, 133, 225, 93, 254, 154, 6, 174, 243, 54, 39, 66, 17, 201, 74, 36, 63, 187, 66, 236, 158, 111, 179, 34, 222, 192, 158, 114, 111, 196, 82, 121, 98, 27, 229, 63, 75, 24, 170, 57, 151, 196, 105, 220, 23, 214, 199, 175, 143, 243, 5, 0, 0, 255, 255, 32, 108, 18, 108, 62, 68, 31,
If I add decoder = new TextDecoder("utf8"); and log(decoder.decode(jsonx)); I get
string=> x�E��xڜ��n\7����'5���*
���$pƋ Ȼ�*��g��#����|�����������������v\�//�_������������
but, HOW TO RETREIVE the array or raw data that I could json.parse ????
If I decompress your data twice, I get:
{"error":null,"result":{"status":"success"},"id":12312}
It looks like you compressed instead of decompressed. Use pako.inflate().
I just had an interview and I have been asked a question: How would you print all the ASCII table characters without using a loop. The language doesn't matter.
The only method for doing so that comes to my mind is by using recursion instead of loops. An algorithm for doing this will be something like:
void printASCII(int i){
if(i == 128)
return;
print(i + " " + ((char)i) + "\n");
printASCII(i + 1);
}
You should call the previous function using:
printASCII(0);
This will print the complete ASCII table, where each line contains the index followed by a space and the actual ASCII character.
I don't think you can find any other way to do so, specially that it clearly says:
The language doesn't matter
This usually means that the question is about an algorithmic idea, rather than being specific for any language.
Two other approaches that weren't mentioned:
The obvious:
#include <stdio.h>
int main() {
printf("%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c", 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127);
}
The power of two tree (probably intended by the interviewer):
#include <stdio.h>
int c;
#define a128 a64; a64;
#define a64 a32; a32;
#define a32 a16; a16;
#define a16 a8; a8;
#define a8 a4; a4;
#define a4 a2; a2;
#define a2 a; a;
#define a printf("%c", c++);
int main() {
c = 0;
a128
}
I was wondering if there is a faster way to randomly choose a value from a list and deleting this value from this list so it cannot be chosen again. This drawing of a value will continue until there aint no values left anymore.
The way I did it soved the problem but it takes almost 8 seconds. So I'm wondering if there is a faster way. I am using Jupyter notebook through the Anaconda software. Since this goes through a server, could it be the problem?
This is what I did:
TotalNumbcol = 266
Column_Numbers = list(np.arange(1,TotalNumbcol+1,1)) # creating a list with all column numbers in it from which can be drawn.
#print Column_Numbers
ABC = Column_Numbers # Creating a variable for the len command in the for loop below, since the Column Numbers length will change.
Chosen_Columns = [[0] for i in range(0,len(Column_Numbers))]
for i in range(len(ABC)):
RandChoiceCol = int(random.choice(Column_Numbers)) # chosing a random number from the Column_Numbers range
Chosen_Columns[i]=(RandChoiceCol) # adding each randomly chosen column number to a list in list showing which column has been chosen.
Column_Numbers = [x for x in Column_Numbers if x not in Chosen_Columns] # delete chosen_column from RandChoiceCol
print Chosen_Columns
print Column_Numbers
[21, 131, 145, 218, 153, 60, 201, 15, 158, 189, 230, 210, 18, 103, 69, 76, 226, 180, 67, 187, 238, 20, 157, 24, 48, 11, 47, 117, 101, 51, 122, 155, 109, 225, 86, 243, 146, 30, 58, 7, 66, 132, 22, 110, 1, 142, 234, 245, 266, 129, 232, 39, 184, 49, 114, 182, 162, 144, 92, 126, 5, 254, 150, 102, 135, 173, 36, 52, 42, 26, 228, 63, 17, 8, 163, 40, 78, 174, 222, 205, 183, 140, 221, 70, 125, 72, 247, 237, 64, 246, 185, 130, 248, 90, 197, 53, 107, 77, 108, 256, 207, 139, 176, 192, 2, 164, 4, 124, 241, 113, 188, 178, 235, 265, 190, 212, 99, 175, 79, 231, 257, 202, 50, 242, 181, 46, 161, 133, 104, 28, 251, 213, 204, 59, 149, 252, 179, 43, 137, 195, 160, 220, 119, 74, 87, 255, 98, 208, 105, 239, 170, 203, 167, 136, 250, 134, 32, 165, 229, 9, 258, 13, 141, 240, 262, 34, 227, 148, 41, 111, 54, 71, 61, 94, 249, 29, 75, 10, 193, 152, 73, 123, 65, 6, 116, 68, 91, 56, 25, 233, 156, 261, 35, 171, 211, 215, 186, 154, 138, 200, 44, 112, 57, 166, 120, 147, 89, 31, 106, 118, 199, 198, 81, 223, 83, 12, 214, 45, 121, 244, 95, 168, 55, 37, 206, 263, 93, 196, 115, 169, 217, 236, 82, 143, 96, 33, 209, 14, 100, 216, 128, 259, 219, 151, 16, 177, 159, 23, 38, 84, 80, 27, 19, 264, 62, 85, 127, 97, 224, 172, 191, 88, 253, 3, 260, 194]
[]
If there is a more efficient way saving time please let me know.
Regards,
You may just shuffle the list in place instead of creating a new list and a new number every time:
Column_Numbers = list(np.arange(1,TotalNumbcol+1,1))
random.shuffle(Column_Numbers)
while Column_Numbers:
rand = Column_Numbers.pop()
print(rand)
I am trying to process some data and write the output in such a way that the result is partitioned by a key, and is sorted by another parameter- say ASC. For example,
>>> data =sc.parallelize(range(10000))
>>> mapped = data.map(lambda x: (x%2,x))
>>> grouped = mapped.groupByKey().partitionBy(2).map(lambda x: x[1] ).saveAsTextFile("mymr-output")
$ hadoop fs -cat mymr-output/part-00000 |cut -c1-1000
[0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66, 68, 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98, 100, 102, 104, 106, 108, 110, 112, 114, 116, 118, 120, 122, 124, 126, 128, 130, 132, 134, 136, 138, 140, 142, 144, 146, 148, 150, 152, 154, 156, 158, 160, 162, 164, 166, 168, 170, 172, 174, 176, 178, 180, 182, 184, 186, 188, 190, 192, 194, 196, 198, 200, 202, 204, 206, 208, 210, 212, 214, 216, 218, 220, 222, 224, 226, 228, 230, 232, 234, 236, 238, 240, 242, 244, 246, 248, 250, 252, 254, 256, 258, 260, 262, 264, 266, 268, 270, 272, 274, 276, 278, 280, 282, 284, 286, 288, 290, 292, 294, 296, 298, 300, 302, 304, 306, 308, 310, 312, 314, 316, 318, 320, 322, 324, 326, 328, 330, 332, 334, 336, 338, 340, 342, 344, 346, 348, 350, 352, 354, 356, 358, 360, 362, 364, 366, 368, 370, 372, 374, 376, 378, 380, 382, 384, 386, 388, 390, 392, 394, 396, 398, 400, 402, 404, 406, 408, 410, 412, 414, 416, 418, 420,
$ hadoop fs -cat mymr-output/part-00001 |cut -c1-1000
[2049, 2051, 2053, 2055, 2057, 2059, 2061, 2063, 2065, 2067, 2069, 2071, 2073, 2075, 2077, 2079, 2081, 2083, 2085, 2087, 2089, 2091, 2093, 2095, 2097, 2099, 2101, 2103, 2105, 2107, 2109, 2111, 2113, 2115, 2117, 2119, 2121, 2123, 2125, 2127, 2129, 2131, 2133, 2135, 2137, 2139, 2141, 2143, 2145, 2147, 2149, 2151, 2153, 2155, 2157, 2159, 2161, 2163, 2165, 2167, 2169, 2171, 2173, 2175, 2177, 2179, 2181, 2183, 2185, 2187, 2189, 2191, 2193, 2195, 2197, 2199, 2201, 2203, 2205, 2207, 2209, 2211, 2213, 2215, 2217, 2219, 2221, 2223, 2225, 2227, 2229, 2231, 2233, 2235, 2237, 2239, 2241, 2243, 2245, 2247, 2249, 2251, 2253, 2255, 2257, 2259, 2261, 2263, 2265, 2267, 2269, 2271, 2273, 2275, 2277, 2279, 2281, 2283, 2285, 2287, 2289, 2291, 2293, 2295, 2297, 2299, 2301, 2303, 2305, 2307, 2309, 2311, 2313, 2315, 2317, 2319, 2321, 2323, 2325, 2327, 2329, 2331, 2333, 2335, 2337, 2339, 2341, 2343, 2345, 2347, 2349, 2351, 2353, 2355, 2357, 2359, 2361, 2363, 2365, 2367, 2369, 2371, 2373, 2375, 2377, 2379, 238
$
Which is perfect- satisfies my first criteria, which is to have results partitioned by key. But I want the result sorted. I tried sorted(), but it didn't work.
>>> grouped= sorted(mapped.groupByKey().partitionBy(2).map(lambda x: x[1] ))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'PipelinedRDD' object is not iterable
I don't want to use parallelize again, and go recursive. Any help would be greatly appreciated.
PS: I did go through this: Does groupByKey in Spark preserve the original order? but it didn't help.
Thanks,
Jeevan.
Yes, that's an RDD, not a Python object that you can sort as if it's a local collection. After groupByKey() though, the value in each key-value tuple is a collection of numbers, and that is what you want to sort? You can use mapValues() which called sorted() on its argument.
I realize it's a toy example but be careful with groupByKey as it has to get all values for a key in memory. Also it is not even necessarily guaranteed that an RDD with 2 elements, and 2 partitions, means 1 goes in each partition. It's probable but not guarnateed.
PS you should be able to replace map(lambda x: x[1]) with values(). It may be faster.
Similar to what is said above the value in key-value is an RDD collection; you can test this by checking type(value). However, you can access a python list via the member .data and call sort or sorted on that.
grouped = mapped.groupByKey().partitionBy(2).map(lambda x: sorted(x[1].data) )
I have just downloaded windows phone 7.1 sdk for windows phone 7.5.
I have two problems:
1. When i create a new project I doesn't see the option to choose windows phone 7 or 7.5
2. Windows phone 7.5 should be based on Silver light 4 but when I create label at my application and write hebrew words it's backwords.
Can I solve the problem at this version?
If you have installed the Mango tools, once you select to create a WP7 application you are prompted on a secondary screen to select whether to target 7.0 or 7.1.
Note that the version number of the SDK and tools is 7.1 but phones running Mango are marketed as version 7.5.
Note that WP7[.1|5] does not yet currently provide native support for RTL languages. You may find a workaround at http://www.danielmoth.com/Blog/RTL-Arabic-And-Hebrew-Support-For-Windows-Phone-7.aspx
For Windows Phone 7 i wrote a mapping function which maps the Arabic encoding[windows-1256] into default WP7 encoding.
public static string ConvertToArabic(string s) { short[] mapping = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 8364, 1662, 8218, 402, 8222, 8230, 8224, 8225, 710, 8240, 1657, 8249, 338, 1670, 1688, 1672, 1711, 8216, 8217, 8220, 8221, 8226, 8211, 8212, 1705, 8482, 1681, 8250, 339, 8204, 8205, 1722, 160, 1548, 162, 163, 164, 165, 166, 167, 168, 169, 1726, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 1563, 187, 188, 189, 190, 1567, 1729, 1569, 1570, 1571, 1572, 1573, 1574, 1575, 1576, 1577, 1578, 1579, 1580, 1581, 1582, 1583, 1584, 1585, 1586, 1587, 1588, 1589, 1590, 215, 1591, 1592, 1593, 1594, 1600, 1601, 1602, 1603, 224, 1604, 226, 1605, 1606, 1607, 1608, 231, 232, 233, 234, 235, 1609, 1610, 238, 239, 1611, 1612, 1613, 1614, 244, 1615, 1616, 247, 1617, 249, 1618, 251, 252, 8206, 8207, 1746 };
string str = string.Empty;
for (int ix = 0; ix < s.Length; ++ix)
{
str = str + (char)mapping[s[ix]];
}
return str;
}
This worked for me well.