When I drop a breakpoint somewhere in method processLine, debugger does not stop at the line. It executes as if there is not any breakpoint .Is debugging akka streams flows somewhat different, how can i solve this issue?
val stream = source.
map( csvLine => A.processLine(csvLine)).
runWith(Sink)
I have had similar issues with ScalaIDE.
My solution has generally been to isolate my "business logic" from any akka dependencies:
//no akka imports required
case class Tweet(val author : String, val body : String)
def validAuthor(author : String) : Boolean = {
author.trim().size > 0 && !author.equalsIgnoreCase("jk_rowling") //breakpoint works here
}
Then my asynchronous code becomes simple calls to the buz logic:
import akka.stream.scaladsl.{Source, Sink}
val source : Source[Tweet,_] = ???
val flow = source.filter(validAuthor)
.runWith(Sink foreach println)
The IDE then obeys breakpoints in the buz logic, e.g. within validAuthor.
Related
I understand eval string-to-function is impossible to use on the browsers' application programming interfaces, but there must be another strategy to use third party dependencies without node.js on v8 engine, given Cloudflare does it in-house, unless they disable the exclusive method by necessity or otherwise on their edge servers for Workers. I imagine I could gather the AST of the commonjs module, as I was able to by rollup watch, but what might the actual steps be, by tooling? I mention AMD for it seems to rely on string-to-function (to-which I've notice Mozilla MDN says nothing much about it).
I have been exploring the require.js repositories, and they either use eval or AST
function DEFNODE(type, props, methods, base) {
if (arguments.length < 4) base = AST_Node;
if (!props) props = [];
else props = props.split(/\s+/);
var self_props = props;
if (base && base.PROPS) props = props.concat(base.PROPS);
var code = "return function AST_" + type + "(props){ if (props) { ";
for (var i = props.length; --i >= 0; ) {
code += "this." + props[i] + " = props." + props[i] + ";";
}
var proto = base && new base();
if ((proto && proto.initialize) || (methods && methods.initialize))
code += "this.initialize();";
code += "}}";
//constructor
var cnstor = new Function(code)();
if (proto) {
cnstor.prototype = proto;
cnstor.BASE = base;
}
if (base) base.SUBCLASSES.push(cnstor);
cnstor.prototype.CTOR = cnstor;
cnstor.PROPS = props || null;
cnstor.SELF_PROPS = self_props;
cnstor.SUBCLASSES = [];
if (type) {
cnstor.prototype.TYPE = cnstor.TYPE = type;
}
if (methods)
for (i in methods)
if (HOP(methods, i)) {
if (/^\$/.test(i)) {
cnstor[i.substr(1)] = methods[i];
} else {
cnstor.prototype[i] = methods[i];
}
}
//a function that returns an object with [name]:method
cnstor.DEFMETHOD = function (name, method) {
this.prototype[name] = method;
};
if (typeof exports !== "undefined") exports[`AST_${type}`] = cnstor;
return cnstor;
}
var AST_Token = DEFNODE(
"Token",
"type value line col pos endline endcol endpos nlb comments_before file raw",
{},
null
);
https://codesandbox.io/s/infallible-darwin-8jcl2k?file=/src/mastercard-backbank/uglify/index.js
https://www.youtube.com/watch?v=EF7UW9HxOe4
Is it possible to make a C++ addon just to add a default object for
node.js named exports or am I Y’ing up the wrong X
'.so' shared library for C++ dlopen/LoadLibrary (or #include?)
“I have to say that I'm amazed that there is code out there that loads one native addon from another native addon! Is it done by acquiring and then calling an instance of the require() function, or perhaps by using uv_dlopen() directly?”
N-API: An api for embedding Node in applications
"[there is no ]napi_env[ just yet]."
node-api: allow retrieval of add-on file name - Missing module in Init
Andreas Rossberg - is AST parsing, or initialize node.js abstraction for native c++, enough?
v8::String::NewFromUtf8(isolate, "Index from C++!");
Rising Stack - Node Source
"a macro implicit" parameter - bridge object between
C++ and JavaScript runtimes
extract a function's parameters and set the return value.
#include <nan.h>
int build () {
NAN_METHOD(Index) {
info.GetReturnValue().Set(
Nan::New("Index from C++!").ToLocalChecked()
);
}
}
// Module initialization logic
NAN_MODULE_INIT(Initialize) {
/*Export the `Index` function
(equivalent to `export function Index (...)` in JS)*/
NAN_EXPORT(target, Index);
}
New module "App" Initialize function from NAN_MODULE_INIT (an atomic?-macro)
"__napi_something doesn't exist."
"node-addon-API module for C++ code (N-API's C code's headers)"
NODE_MODULE(App, Initialize);
Sep 17, 2013, 4:42:17 AM to v8-u...#googlegroups.com "This comes up
frequently, but the answer remains the same: scrap the idea. ;)
Neither the V8 parser nor its AST are designed for external
interfacing. In particular (1) V8's AST does not necessarily reflect
JavaScript syntax 1-to-1, (2) we change it all the time, and (3) it
depends on various V8 internals. And since all these points are
important for V8, don't expect the situation to change.
/Andreas"
V8 c++: How to import module via code to script context (5/28/22, edit)
"The export keyword may only be used in a module interface unit.
The keyword is attached to a declaration of an entity, and causes that
declaration (and sometimes the definition) to become visible to module
importers[ - except for] the export keyword in the module-declaration, which is just a re-use of the keyword (and does not actually “export” ...entities)."
SyntheticModule::virtual
ScriptCompiler::CompileModule() - "Corresponds to the ParseModule abstract operation in the ECMAScript specification."
Local<Function> foo_func = ...;//external
Local<Module> module = Module::CreateSyntheticModule(
isolate, name,
{String::NewFromUtf8(isolate, "foo")},
[](Local<Context> context, Local<Module> module) {
module->SetSyntheticModuleExport(
String::NewFromUtf8(isolate, "foo"), foo_func
);
});
Context-Aware addons from node.js' commonjs modules
export module index;
export class Index {
public:
const char* app() {
return "done!";
}
};
import index;
import <iostream>;
int main() {
std::cout << Index().app() << '\n';
}
node-addon-api (new)
native abstractions (old)
"Thanks to the crazy changes in V8 (and some in Node core), keeping native addons compiling happily across versions, particularly 0.10 to 0.12 to 4.0, is a minor nightmare. The goal of this project is to store all logic necessary to develop native Node.js addons without having to inspect NODE_MODULE_VERSION and get yourself into a macro-tangle[ macro = extern atomics?]."
Scope Isolate (v8::Isolate), variable Local (v8::Local)
typed_array_to_native.cc
"require is part of the Asynchronous Module Definition AMD API[, without "string-to-function" eval/new Function()],"
node.js makes objects, for it is written in C++.
"According to the algorithm, before finding
./node_modules/_/index.js, it tried looking for express in the
core Node.js modules. This didn’t exist, so it looked in node_modules,
and found a directory called _. (If there was a
./node_modules/_.js, it would load that directly.) It then
loaded ./node_modules/_/package.json, and looked for an exports
field, but this didn’t exist. It also looked for a main field, but
this didn’t exist either. It then fell back to index.js, which it
found. ...require() looks for node_modules in all of the parent directories of the caller."
But java?
I won't accept this answer until it works, but this looks promising:
https://developer.oracle.com/databases/nashorn-javascript-part1.html
If not to run a jar file or something, in the Worker:
https://github.com/nodyn/jvm-npm
require and build equivalent in maven, first, use "dist/index.js".
Specifically: [ScriptEngineManager][21]
https://stackoverflow.com/a/15787930/11711280
Actually: js.commonjs-require experimental
https://docs.oracle.com/en/graalvm/enterprise/21/docs/reference-manual/js/Modules/
Alternatively/favorably: commonjs builder in C (v8 and node.js)
https://www.reddit.com/r/java/comments/u7elf4/what_are_your_thoughts_on_java_isolates_on_graalvm/
Here I will explore v8/node.js src .h and .cc for this purpose
https://codesandbox.io/s/infallible-darwin-8jcl2k?file=/src/c.cpp
I'm curious why there is near machine-level C operability in Workers if not to use std::ifstream, and/or build-locally, without node.js require.
I am trying to upgrade from version 0.18 to 0.19 of Elm. My project depends on elm-lang/websocket in 0.18? I cannot seem to find the equivalent package in 0.19. What am I missing?
Here is a minimal working example of an interactive form to echo input from echo.websocket.org using 2 simple input/output ports for communicating with a JavaScript WebSocket object external to the elm 0.19 module:
File: echo.elm. Compile with: elm make echo.elm --output=echo.js
port module Main exposing (main)
import Browser
import Html exposing (Html)
import Html.Attributes as HA
import Html.Events as HE
import Json.Encode as JE
-- JavaScript usage: app.ports.websocketIn.send(response);
port websocketIn : (String -> msg) -> Sub msg
-- JavaScript usage: app.ports.websocketOut.subscribe(handler);
port websocketOut : String -> Cmd msg
main = Browser.element
{ init = init
, update = update
, view = view
, subscriptions = subscriptions
}
{- MODEL -}
type alias Model =
{ responses : List String
, input : String
}
init : () -> (Model, Cmd Msg)
init _ =
( { responses = []
, input = ""
}
, Cmd.none
)
{- UPDATE -}
type Msg = Change String
| Submit String
| WebsocketIn String
update : Msg -> Model -> (Model, Cmd Msg)
update msg model =
case msg of
Change input ->
( { model | input = input }
, Cmd.none
)
Submit value ->
( model
, websocketOut value
)
WebsocketIn value ->
( { model | responses = value :: model.responses }
, Cmd.none
)
{- SUBSCRIPTIONS -}
subscriptions : Model -> Sub Msg
subscriptions model =
websocketIn WebsocketIn
{- VIEW -}
li : String -> Html Msg
li string = Html.li [] [Html.text string]
view : Model -> Html Msg
view model = Html.div []
--[ Html.form [HE.onSubmit (WebsocketIn model.input)] -- Short circuit to test without ports
[ Html.form [HE.onSubmit (Submit model.input)]
[ Html.input [HA.placeholder "Enter some text.", HA.value model.input, HE.onInput Change] []
, model.responses |> List.map li |> Html.ol []
]
]
Embed the compiled echo.js into echo.html:
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Echo</title>
<script src="echo.js"></script>
</head>
<body>
<div id="elm-node"></div>
<script>
var app = Elm.Main.init({node: document.getElementById("elm-node")});
var ws = new WebSocket("wss://echo.websocket.org");
ws.onmessage = function(message)
{
console.log(message);
app.ports.websocketIn.send(JSON.stringify({data:message.data,timeStamp:message.timeStamp}));
};
app.ports.websocketOut.subscribe(function(msg) { ws.send(msg); });
</script>
</body>
</html>
This works on Firefox 60.2.0esr on Linux but has not been tested on other platforms.
Again, this is only a minimal example to demonstrate how to use ports with WebSockets for Elm 0.19. It does not included closing the WebSocket, error handling, etc. but hopefully this example can help you get started in that direction. It is expected that WebSockets will be directly supported by Elm again soon, so this is only a temporary work-around. If you don't need to upgrade to 0.19, then consider staying with 0.18 instead.
The websocket package is currently redesigned for Elm 0.19, see this issue:
This package has not been updated for 0.19 yet. I have heard lots of folks saying they need more features from this package, so I'd rather take that into consideration in the update rather than just doing the same stuff. I recommend using ports or 0.18 if you absolutely need this right this second.
EDIT: April 15, 2020 update
The package has been archived and the Readme file updated as
follows:
The recommended way to use WebSockets with Elm for now is through
ports. You can see a minimal example in the js-integration-examples
repo [IMAGE CLIPPED]
History
We had a bare bones version of WebSockets in within Elm in versions
0.17 and 0.18, part of the introduction of subscriptions to Elm. But users found that the API was not able to cover a lot of situations
they faced in practice. How can this work with Elixir Pheonix?
Firebase? How can I use a different backoff strategy for reconnecting?
How can I hear about when the connection goes down or comes back? How
about sub-protocols?
In trying to expand the API to cover all the cases people were facing
in practice, I came to think that it may not be possible with the
current subscriptions infrastructure. (My feeling is that effect
managers may not be a great fit for web sockets because they do not
have great mechanisms for uniquely identifying resources. Do we have
one connections or two? How do we tell the difference? If it requires
function pointer equality, how can we make that reliable when an
anonymous function is used?) I did not understand this problem as well
in 2016, and I think it manifested most clearly with web sockets.
So facing the prospect of either (1) having an API that many
eventually had to leave behind for ports or (2) recommending that
people go with ports from the start, we figured that (2) was probably
the best for someone new coming to Elm. That way they would hook up to
their preferred web socket manager without the intermediate step of
learning a promising but incomplete API.
My board (apf27) has a processor (i.MX27) and a FPGA (Spartan3A) that communicate through a "memory bus" called WEIM in proc datasheet.
I want to transfer data from the FPGA to the processor. I managed to do it with a simple Output() IO :
val io = IO(new Bundle {
...
val data = Output(UInt(16.W))
val oen = Input(Bool())
...
I can read data from the processor, but that "lock" the bus. I have to release it for the nand component also present on it.
To release it I can use the signal oen (output enable) but I can't assign a high impedance value like 'Z' in Verilog/VHDL to 'release' it.
What is the right way to do it in Chisel3 ? I saw something called 'AnalogRawModule" in chisel3 github is it the things to use ?
Analog is what you're looking for. It is basically an escape to allow bidirectional wires and other signals that aren't really supported by Chisel to still connect through your Chisel design.
Here's an example:
import chisel3._
import chisel3.experimental.Analog
class AnalogBlackBox extends BlackBox {
val io = IO(new Bundle {
val bus = Analog(32.W)
})
}
class AnalogModule extends Module {
val io = IO(new Bundle {
val bus = Analog(32.W)
})
val inst = Module(new AnalogBlackBox)
inst.io.bus <> io.bus
}
object AnalogDriver extends App {
chisel3.Driver.execute(args, () => new AnalogModule)
}
You can't drive Analog-type wires in Chisel and unfortunately you can't do concatenation or bit select (although we should support that), but you can at least connect signals through. If you need to do any kind of bit selection or concatenation, you need to do that in a BlackBox.
I'm using Elastic4s (Scala Client for ElasticSearch).
I can retrieve multiGet results with await :
val client = HttpClient(ElasticsearchClientUri(esHosts, esPort))
val resp = client.execute {
multiget(
get(C1) from "mlphi_crm_0/profiles" fetchSourceInclude("crm_events"),
get(C2) from "mlphi_crm_0/profiles" fetchSourceInclude("crm_events"),
get(C3) from "mlphi_crm_0/profiles" fetchSourceInclude("crm_events")
)
}.await
val result = resp.items
But I've read that in practice it's better to avoid this ".await".
How can we do that ? thanks
You shouldn't use .await because you're blocking the thread waiting for the future to return.
Instead you should handle the future like you would any other API that returns futures - whether that be reactive-mongo, akka.ask or whatever.
I realise this is old, but in case anyone else comes across it, the simplest way to handle this would be:
val client = HttpClient(ElasticsearchClientUri(esHosts, esPort))
val respFuture = client.execute {
multiget(
get(C1) from "mlphi_crm_0/profiles" fetchSourceInclude("crm_events"),
get(C2) from "mlphi_crm_0/profiles" fetchSourceInclude("crm_events"),
get(C3) from "mlphi_crm_0/profiles" fetchSourceInclude("crm_events")
)
}
respFuture.map(resp=> ...[do stuff with resp.items])
The key thing here is that your processing actually takes place in a subthread, which Scala takes care of calling for you when, any only when, the data is ready for you. The caller keeps running immediately after respFuture.map(). Whatever your function in map(()=>{}) returns is passed back as a new Future; if you don't need it then use onComplete or andThen as they make error handling a little easier.
See https://docs.scala-lang.org/overviews/core/futures.html for more details on Futures handling, and https://alvinalexander.com/scala/concurrency-with-scala-futures-tutorials-examples for some good examples
I'm trying to set up a simple example using WebSockets in Elm, but I keep getting the run time error "input is undefined". The console does not give me any line number in my elm file or anything like that.
I was trying to use WebSockets in a large project, and I kept getting the error "a is undefined", so I decided to make this small example to try and isolate the problem.
I wrote some code that receives messages containing numbers from the websocket. It increments the numbers, and then sends the new numbers back out over the web socket. The server does the same thing, sending back the number incremented by 1 to the client.
Here is the elm code:
import Graphics.Element (Element)
import Signal
import Signal (Signal)
import Text
import Window
import WebSocket
import String
type State = Num Int
| StateErr String
input : Signal String
input =
WebSocket.connect "ws://localhost:4567/test" sendToServer
sendToServer : Signal String
sendToServer =
Signal.dropRepeats
(Signal.dropIf (\str -> str == "") "" (Signal.map formatReply state))
formatReply : State -> String
formatReply state =
case state of
Num n -> toString n
StateErr str -> ""
stepState : String -> State -> State
stepState str state =
case (String.toInt str) of
Ok n -> Num (n + 1)
Err str -> StateErr str
display : (Int,Int) -> State -> Element
display (w,h) state = Text.asText state
state : Signal State
state =
Signal.foldp stepState (Num 0) input
main : Signal Element
main =
Signal.map2 display Window.dimensions state
I tested the server side, and it's working fine, so I definitely do not think that the server is causing the issue.
When I tried the code in Firefox, I get "input is undefined". When I run it in Chrome, I get "Cannot read property 'kids' of undefined".
In Chrome, upon looking at the stack trace it seems that when the code goes to run, input is undefined. Is this a bug with the WebSocket library?
I'm very new to using Elm, so I'd appreciate any help/advice on using websockets.
I learned that the cause of my troubles, is that as of now the WebSockets library in elm is not fully implemented. I also learned that I can accomplish my goals using ports, and then implementing the websocket in javascript.
I added the following javascript to my html file:
var game = Elm.fullscreen(Elm.SlimeWarz, {rawServerInput: ""});
var socket = new WebSocket("ws://localhost:4567");
socket.onopen = function(){
console.log("Socket has been opened.");
}
socket.onmessage = function(msg){
game.ports.rawServerInput.send(msg.data);
}
game.ports.sendToServer.subscribe(sendOverWebsocket);
function sendOverWebsocket(str) {
socket.send(str);
}
Then in elm, I can send data using a ported Signal called sendtoServer
port sendToServer : Signal String
and I can view all the data I receive through the ported signal rawServerInput
port rawServerInput : Signal String
Answer
I'm going to use part of my answer to this question. I think your solution to use ports and do the websocket part in JavaScript is better than the hack I described in that answer, but you may still want to look at it. Once Elm 0.15 is released this problem should go away entirely because of a language feature and the revamp of Websocket (Http gets revamped too btw).
Context: Reason for error
The reason you get the runtime error is because of a compiler bug. The compiler only generates correct code on recursive functions, while it accepts any recursive value. Notice that your input depends on sendToServer, sendToserver depends on state and sContext: tate depends on input.
Context: Code architecture
Those kinds of cyclic signals are usually a sign of bad program architecture. You can find more on that subject here. But your architecture is not at fault here. The problem lies with the Websocket library which doesn't steer you in the right direction.