EventEmitter in Nestjs is wrapper around EventEmitter2 module. I whant that Server-Sent Events return Observable with EE.
import { Controller, Post, Body, Sse } from '#nestjs/common';
import { fromEvent } from 'rxjs';
import { EventEmitter2 } from '#nestjs/event-emitter';
import { OrdersService } from './orders.service';
import { CreateOrderDto } from './dto/create-order.dto';
#Controller('orders')
export class OrdersController {
constructor(private ordersService: OrdersService,
private eventEmitter2: EventEmitter2) {}
#Post()
createOrder(#Body() createOrderDto: CreateOrderDto) {
// save `Order` in Mongo
const newOrder = this.ordersService.save(createOrderDto);
// emit event with new order
this.eventEmitter2.emit('order.created', newOrder);
return newOrder;
}
#Sse('newOrders')
listenToTheNewOrders() {
// return Observable from EventEmitter2
return fromEvent(this.eventEmitter2, 'order.created');
}
}
But after subscribtion to this source from browser i've getting only errors
this.eventSource = new EventSource('http://localhost:3000/api/v1/orders/newOrders');
this.eventSource.addEventListener('open', (o) => {
console.log("The connection has been established.");
});
this.eventSource.addEventListener('error', (e) => {
console.log("Some erorro has happened");
console.log(e);
});
this.eventSource.addEventListener('message', (m) => {
const newOder = JSON.parse(m.data);
console.log(newOder);
});
It's quite likely that you forgot to format the event in the right way.
For SSE to work internally, each chunk needs to be a string of such format: data: <your_message>\n\n - whitespaces do matter here. See MDN reference.
With Nest.js, you don't need to create such message manually - you just need to return a JSON in the right structure.
So in your example:
#Sse('newOrders')
listenToTheNewOrders() {
// return Observable from EventEmitter2
return fromEvent(this.eventEmitter2, 'order.created');
}
would have to be adjusted to, for example:
#Sse('newOrders')
listenToTheNewOrders() {
// return Observable from EventEmitter2
return fromEvent(this.eventEmitter2, 'order.created')
.pipe(map((_) => ({ data: { newOrder } })));
}
the structure { data: { newOrder } } is key here. This will be later translated by Nest.js to earlier mentioned data: ${newOrder}\n\n
I'm learning how to fetch asynchronous data in a web component using the fetch API and lit-element:
import {LitElement, html} from 'lit-element';
class WebIndex extends LitElement {
connectedCallback() {
super.connectedCallback();
this.fetchData();
}
fetchData() {
fetch('ajax_url')
.then(response => {
if (!response.ok) {
throw new Error('Network response was not ok');
};
response.json();
})
.then(data => {
this.data = data;
console.log('Success:', data);
})
.catch((error) => {
console.error('Error:', error);
});
}
render() {
if (!this.data) {
return html`
<h4>Loading...</h4>
`;
}
return html`
<h4>Done</h4>
`;
}
}
customElements.define('web-index', WebIndex);
However the html rendered never changes. What I'm doing wrong? Is this the best way to fetch asynchronous data in a web component?
You need to register data in component properties so that the render is called once value of data is changed
static get properties() {
return {
data: Object
}
}
https://lit-element.polymer-project.org/guide/properties
My code are:-
function showData(data) {
return {
type: 'SHOWDATA',
data,
};
}
export default {
fetchData() {
return function (dispatch) {
getDataApi.getData().then((response)=>dispatch(showData(response)).catch()
};},
updateData{
return function (dispatch) {
getDataApi.getData().then((response)=>if(response.isSucess)
{dispatch(fetchData())}).catch()
};}
}
After update call of the action I want to refresh the list thats why I
called dispatch(fetchData()); but it is showing that fetchData not
defined.How can I call the method defined in same export default function.
Can this help you? Not really exported as default but its named.
export const Actions = {
getAll,
add,
update,
view,
search
}
function getAll(){
return dispatch => {
dispatch(request());
Service.getAll()
.then(
response => {
// todo...
},
error => {
// catch error
}
);
}
function request() { return { type: Constants.LIST_REQUEST } }
function success(data) { return { type: Constants.LIST_SUCCESS, data } }
function failure(error) { return { type: Constants.LIST_FAILURE, error } }
}
function add(data){
return dispatch => {
dispatch(request());
Service.add(data)
.then(
response => {
if(response.status === 'fail'){
// do something
}else{
dispatch(success(response));
dispatch(getAll());
}
},
error => {
// do something
}
);
}
function request() { return { type: Constants.ADD_REQUEST } }
function success(data) { return { type: Constants.ADD_SUCCESS, data } }
function failure(error) { return { type: Constants.ADD_FAILURE, error } }
}
So I am learning React-Redux. I am trying to find a way to load a value from the database first on load instead of just starting from 0. I have written an action that will hit an endpoint on my node file and pull from my mongo database. This works. However the action never seems to reach the reducer to actually update the store. Can someone explain to me the right way to make sure this action is store aware.
Here is the code for the action. Note the console.log with the number prints out what I want. I just never see the logs in the reducer that it was even ever reached.
export function setFooClicks(){
console.log("in the set foo clicks action")
var number = 0;
//return function(dispatch){
//console.log("in the return function")
return axios.get('/clicks').then(result => {
number = result.data
console.log("The number of clicks is", number)
//return number
return{
type: types.SETFOOCLICKS,
totalFoo: result.data
}
}).catch((error) => {
return console.log(error);
})
//}
}
I am trying to grab it in the top level container at the moment so here is the code for that.
import React, { Component } from 'react'
import { connect } from 'react-redux'
import Foo from '../components/Foo'
import { incrementFoo, setFooClicks } from '../actions'
class FooContainer extends Component {
componentDidMount(){
setFooClicks();
}
render() {
return (
<div>
<Foo incrementFooAction={() => this.props.incrementFoo()} totalFoo={this.props.totalFoo}/>
</div>
)
}
}
function mapStateToProps(state) {
return {
totalFoo: state.foo.totalFoo
}
}
export default connect(mapStateToProps, { incrementFoo,setFooClicks })(FooContainer)
The normal incrementFoo action works but trying to add the setFooClicks action into the component as well doesn't. Lastly here is the code to the reducer funciton. All I did was add a case to the switch block.
export default function incrementFoo(state = initialState, action) {
console.log("I am in the foo reducer")
console.log(action.type)
switch (action.type) {
case INCREMENT:
console.log(state.totalFoo);
return {
...state,
totalFoo: state.totalFoo + 1
}
case SETFOOCLICKS:
console.log("in the SETFOOCLICKS reducer")
return{
...state,
totalFoo: action.totalFoo
}
default:
return state
}
}
Using NodeJS elastic search client. Trying to write a data importer to bulk import documents from MongoDB. The problem I'm having is the index refresh doesn't seem to wait until all documents are written to elastic before checking the counts.
Using the streams API in node to read the records into a batch, then using the elastic API bulk command to write the records. Shown below:
function rebuildIndex(modelName, queryStream, openStream, done) {
logger.debug('Rebuilding %s index', modelName);
async.series([
function (next) {
deleteType(modelName, function (err, result) {
next(err, result);
});
},
function (next) {
var Model;
var i = 0;
var batchSize = settings.indexBatchSize;
var batch = [];
var stream;
if (queryStream && !openStream) {
stream = queryStream.stream();
} else if (queryStream && openStream) {
stream = queryStream;
}else
{
Model = mongoose.model(modelName);
stream = Model.find({}).stream();
}
stream.on("data", function (doc) {
logger.debug('indexing %s', doc.userType);
batch.push({
index: {
"_index": settings.index,
"_type": modelName.toLowerCase(),
"_id": doc._id.toString()
}
});
var obj;
if (doc.toObject){
obj = doc.toObject();
}else{
obj = doc;
}
obj = _.clone(obj);
delete obj._id;
batch.push(obj);
i++;
if (i % batchSize == 0) {
console.log(chalk.green('Loaded %s records'), i);
client().bulk({
body: batch
}, function (err, resp) {
if (err) {
next(err);
} else if (resp.errors) {
next(resp);
}
});
batch = [];
}
});
// When the stream ends write the remaining records
stream.on("end", function () {
if (batch.length > 0) {
console.log(chalk.green('Loaded %s records'), batch.length / 2);
client().bulk({
body: batch
}, function (err, resp) {
if (err) {
logger.error(err, 'Failed to rebuild index');
next(err);
} else if (resp.errors) {
logger.error(resp.errors, 'Failed to rebuild index');
next(resp);
} else {
logger.debug('Completed rebuild of %s index', modelName);
next();
}
});
} else {
next();
}
batch = [];
})
}
],
function (err) {
if (err)
logger.error(err);
done(err);
}
);
}
I use this helper to check the document counts in the index. Without the timeout, the counts in the index are wrong, but with the timeout they're okay.
/**
* A helper function to count the number of documents in the search index for a particular type.
* #param type The type, e.g. User, Customer etc.
* #param done A callback to report the count.
*/
function checkCount(type, done) {
async.series([
function(next){
setTimeout(next, 1500);
},
function (next) {
refreshIndex(next);
},
function (next) {
client().count({
"index": settings.index,
"type": type.toLowerCase(),
"ignore": [404]
}, function (error, count) {
if (error) {
next(error);
} else {
next(error, count.count);
}
});
}
], function (err, count) {
if (err)
logger.error({"err": err}, "Could not check index counts.");
done(err, count[2]);
});
}
And this helper is supposed to refresh the index after the update completes:
// required to get results to show up immediately in tests. Otherwise there's a 1 second delay
// between adding an entry and it showing up in a search.
function refreshIndex(done) {
client().indices.refresh({
"index": settings.index,
"ignore": [404]
}, function (error, response) {
if (error) {
done(error);
} else {
logger.debug("deleted index");
done();
}
});
}
The loader works okay, except this test fails because of timing between the bulk load and the count check:
it('should be able to rebuild and reindex customer data', function (done) {
this.timeout(0); // otherwise the stream reports a timeout error
logger.debug("Testing the customer reindexing process");
// pass null to use the generic find all query
searchUtils.rebuildIndex("Customer", queryStream, false, function () {
searchUtils.checkCount("Customer", function (err, count) {
th.checkSystemErrors(err, count);
count.should.equal(volume.totalCustomers);
done();
})
});
});
I observe random results in the counts from the tests. With the artificial delay (setTimeout in the checkCount function) then the counts match. So I conclude that the documents are eventually written to elastic and the test would pass. I thought the indices.refresh would essentially force a wait until the documents are all written to the index, but it doesn't seem to be working with this approach.
The setTimeout hack is not really sustainable when the volume goes to actual production level....so how can I ensure the bulk calls are completely written to elastic index before checking the count of documents?
Take a look at the "refresh" parameter (elasticsearch documentation)
For example:
let bulkUpdatesBody = [ bulk actions / docs to index go here ]
client.bulk({
refresh: "wait_for",
body: bulkUpdatesBody
});
I'm not sure if this is the answer or not - but I flushed the index prior to checking the count. It "appears" to work, but I don't know if it's just because of the timing between the calls. Perhaps someone from elastic team knows if flushing the index will really solve the issue?
function checkCount(type, done) {
async.series([
function(next) {
client().indices.flush({
"index": settings.index,
"ignore": [404]
}, function (error, count) {
if (error) {
next(error);
} else {
next(error, count.count);
}
});
},
function (next) {
refreshIndex(type, next);
},
function (next) {
client().count({
"index": settings.index,
"type": type.toLowerCase(),
"ignore": [404]
}, function (error, count) {
if (error) {
next(error);
} else {
next(error, count.count);
}
});
}
], function (err, count) {
if (err)
logger.error({"err": err}, "Could not check index counts.");
done(err, count[2]);
});
}