Using kue-scheduler with ParseServer on Heroku - heroku

In running kue-scheduler on heroku with the heroku redis plugin, while I can get kue jobs to work, it seems that kue-scheduler is requiring certain configuration of redis not allowed for in the heroku redis environment. Has anyone had success running kue-scheduler in an Heroku environment. Here is the start of my index.js file:
var express = require('express');
var ParseServer = require('parse-server').ParseServer;
var path = require('path');
var kue = require('kue-scheduler')
var queue = kue.createQueue({redis:
'redis://h:***************#ec2-**-19-83-130.compute-1.amazonaws.com:23539'
});
var job = queue.create('test', {
title: 'Hello world'
, to: 'j#example.com'
, template: 'welcome-email'
}).save( function(err){
if( !err ) console.log( job.id );
});
job.log('$Job %s run', job.id);
queue.every('30 seconds', job);
queue.process('test', function(job, done){
test_function(job.data.title, done);
});
function test_function(title, done) {
console.log('Ran test function with title %s', title)
// email send stuff...
done();
}
And here is the error.
2016-07-21T00:46:26.445297+00:00 app[web.1]: /app/node_modules/parse-server/lib/ParseServer.js:410
2016-07-21T00:46:26.445299+00:00 app[web.1]: throw err;
2016-07-21T00:46:26.445300+00:00 app[web.1]: ^
2016-07-21T00:46:26.445417+00:00 app[web.1]: ReplyError: ERR unknown command 'config'
2016-07-21T00:46:26.445419+00:00 app[web.1]: at parseError (/app/node_modules/redis-parser/lib/parser.js:161:12)
2016-07-21T00:46:26.445420+00:00 app[web.1]: at parseType (/app/node_modules/redis-parser/lib/parser.js:222:14)
2016-07-21T00:46:26.466188+00:00 app[web.1]:
The issue is that heroku redis doesn't allow config options on its redis infrastructure from what I can tell.
If someone has had success, grateful for any suggestions.

managed to solve this by:
var queue = kue.createQueue(
{redis: 'redis://xxxxxxxxxxxxx#ec2-50-19-83-130.compute-1.amazonaws.com:23539',
skipConfig: true
});
Just need the skipConfig parameter

I was having the same problem and was unable to get kue-scheduler working on Heroku-Redis. To solve, I instead used the Heroku Add-on Redis Cloud.
This allows you to set the required Redis flag notify-keyspace-events which isn't modifiable on the regular Heroku-Redis add-on. To set this flag:
Add Redis Cloud heroku add-on
Go to heroku settings page
Reveal Config Vars in Config Variables
Copy REDISCLOUD_URL, it should be something like redis://rediscloud:PASSWORD#xxx.redislabs.com:PORT_NUMBER
In terminal enter redis-cli -h xxx.redislabs.com -p PORT_NUMBER -a PASSWORD with variables from REDISCLOUD_URL
Once connected, enter config set notify-keyspace-events Ex
You can verify it is set correctly by entering config get notify-keyspace-events
Make sure to update your javascript code to point to your new REDISCLOUD_URL when calling kue.createQueue()
credit to #josephktcheung for their work though here: https://github.com/lykmapipo/kue-scheduler/issues/46

Related

Shopware 6: Cypress test - reset database failed

I try to cleanup my database with command cy.cleanUpPreviousState:
// mytest.cy.js
...
beforeEach(() => {
cy.cleanUpPreviousState()
})
...
the request was response with error:
CypressError
cy.request() failed trying to load:
http://my-route.dev.localhost:8005/cleanup
The app runs in docker container, using shyim/shopware-docker
Questions
What is wrong with my request/route?
Which controller has to take this request?
To find out what is wrong, have a log at the network tab request log.
Answering your second question: There is a special server spun up for this action. It is not a normal Shopware route.
See in the cypress.js - it is supposed to use psh.phar to clean-up when this URL is called.
const requestedUrl = request.url;
if (requestedUrl !== "/cleanup") {
response.end();
return;
}
return childProcess.exec(
`${PROJECT_ROOT}/psh.phar e2e:cleanup`,
[...]
server.listen(8005);
So things to check are:
Is that port forwarded to your docker container?
Are you using the development template and is psh.phar existing?

Suddenly, Heroku credentials to a PostgreSQL server gives FATAL password for user error

Without changing anything in my settings, I can't connect to my PostgreSQL database hosted on Heroku. I can't access it in my application, and is given error
OperationalError: (psycopg2.OperationalError) FATAL: password authentication failed for user "<heroku user>" FATAL: no pg_hba.conf entry for host "<address>", user "<user>", database "<database>", SSL off
It says SSL off, but this is enabled as I have confirmed in PgAdmin. When attempting to access the database through PgAdmin 4 I get the same problem, saying that there is a fatal password authentication for user '' error.
I have checked the credentials for the database on Heroku, but nothing has changed. Am I doing something wrong? Do I have to change something in pg_hba.conf?
Edit: I can see in the notifications on Heroku that the database was updated right around the time the database stopped working for me. I am not sure if I triggered the update, however.
Here's the notification center:
In general, it isn't a good idea to hard-code credentials when connecting to Heroku Postgres:
Do not copy and paste database credentials to a separate environment or into your application’s code. The database URL is managed by Heroku and will change under some circumstances such as:
User-initiated database credential rotations using heroku pg:credentials:rotate.
Catastrophic hardware failures that require Heroku Postgres staff to recover your database on new hardware.
Security issues or threats that require Heroku Postgres staff to rotate database credentials.
Automated failover events on HA-enabled plans.
It is best practice to always fetch the database URL config var from the corresponding Heroku app when your application starts. For example, you may follow 12Factor application configuration principles by using the Heroku CLI and invoke your process like so:
DATABASE_URL=$(heroku config:get DATABASE_URL -a your-app) your_process
This way, you ensure your process or application always has correct database credentials.
Based on the messages in your screenshot, I suspect you were affected by the second bullet. Whatever the cause, one of those messages explicitly says
Once it has completed, your database URL will have changed
I had the same issue. Thx to #Chris I solved it this way.
This file is in config/database.js (Strapi 3.1.3)
var parseDbUrl = require("parse-database-url");
if (process.env.NODE_ENV === 'production') {
module.exports = ({ env }) => {
var dbConfig = parseDbUrl(env('DATABASE_URL', ''));
return {
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: dbConfig.driver,
host: dbConfig.host,
port: dbConfig.port,
database: dbConfig.database,
username: dbConfig.user,
password: dbConfig.password,
},
options: {
ssl: false,
},
},
},
}
};
} else {
// to use the default local provider you can return an empty configuration
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'sqlite',
filename: env('DATABASE_FILENAME', '.tmp/data.db'),
},
options: {
useNullAsDefault: true,
},
},
},
});
}

Heroku postgres node connection timeout

I'm trying to connect to a Postgres database from my Heroku node app, which works when running locally, both through node and by running the heroku local web command, but when running it on Heroku, it times out while waiting for pool.connect
I'm running the following code snippet through the Heroku console (I've also tried using this code in my app directly, but this is more efficient than redeploying each time):
node -e "
const { Pool } = require('pg');
const pool = new Pool({
connectionTimeoutMillis: 15000,
connectionString: process.env.DATABASE_URL + '?sslmode=require',
ssl: {
rejectUnauthorized: true
}
});
console.log('pool created');
(async() => {
try {
console.log('connecting');
const client = await pool.connect(); // this never resolves
console.log('querying');
const { rows } = await client.query('SELECT * FROM test_table LIMIT 1;');
console.log('query success', rows);
client.release()
} catch (error) {
console.log('query error', error);
}
})()
"
Things I've tried so far:
Using the pg Clientinstead of Pool
Using ssl: true instead of ssl: { rejectUnauthorized: true }
Using client.query without using pool.connect
Increased and omitted connectionTimeoutMillis (it resolves quickly when running locally since I'm querying a database that has just one row)
I've also tried using callbacks and promises instead of async / await
I've tried setting the connectionString both with the ?sslmode=require parameter and without it
I have tried using pg versions ^7.4.1 and ^7.18.2 so far
My assumption is that there is something I'm missing with either the Heroku setup or SSL, any help would be greatly appreciated, Thanks!

Issues running BrowserStackLocal for website behind firewall

I'm trying to run browserstack behind the firewall.
I tried to run this command on terminal:
RK$ ./BrowserStackLocal --key <key> --force-local
BrowserStackLocal v7.0
You can now access your local server(s) in our remote browser.
Press Ctrl-C to exit
I opened another terminal and I ran the command
npm run test:functional:cr:mobile
I get the following error:
1) Run sample test flow page:
Uncaught WebDriverError: [browserstack.local] is set to true but local testing through BrowserStack is not connected.
This is my config.js
'use strict'
import webdriver from 'selenium-webdriver'
let driver
module.exports = {
getDriverConfiguration: function (testTitle, browserName) {
var capabilities = {
'browserName': process.env.BROWSER || 'Chrome',
'realMobile': 'true',
'os': 'android',
'deviceName': process.env.DEVICE || 'Samsung Galaxy S8',
'browserstack.user': 'USER',
'browserstack.key': 'KEY',
'browserstack.debug': 'true',
'build': 'Build for mobile testing',
'browserstack.local' : 'true',
'browserstack.localIdentifier' : 'Test123'
}
driver = new webdriver.Builder().withCapabilities(capabilities).usingServer('http://hub-cloud.browserstack.com/wd/hub').build()
driver.manage().deleteAllCookies()
return driver
}
}
I enabled browserstack.local to true but I still get this error.
Not sure where I'm going wrong.
Please kindly help.
The error [browserstack.local] is set to true but local testing through BrowserStack is not connected. is returned if your BrowserStackLocal connection (the one you established using ./BrowserStackLocal --key --force-local) is disconnected.
I would suggest you use the following approach instead, to avoid the additional step and easily manage your local testing connection:
npm install browserstack-local
Once you have installed the browserstack-local module, use the following code snippet as reference to modify your code and start browserstack-local from your code itself(before the line driver = new webdriver.Builder().withCapabilities(capabilities).usingServer('http://hub-cloud.browserstack.com/wd/hub').build()), instead of starting it from a separate terminal window:
var browserstack = require('browserstack-local');
//creates an instance of Local
var bs_local = new browserstack.Local();
// replace <browserstack-accesskey> with your key. You can also set an environment variable - "BROWSERSTACK_ACCESS_KEY".
var bs_local_args = { 'key': '<browserstack-accesskey>', 'forceLocal': 'true' };
// starts the Local instance with the required arguments
bs_local.start(bs_local_args, function() {
console.log("Started BrowserStackLocal");
});
// check if BrowserStack local instance is running
console.log(bs_local.isRunning());
// stop the Local instance
bs_local.stop(function() {
console.log("Stopped BrowserStackLocal");
});

Sailsjs 0.10.3 - Heroku - RedisToGo - req.session undefined

When using sailsjs v0.10.3 with Redis To Go for session storage, req.session is always undefined.
It is undefined when I deploy both locally and to Heroku. req.session is correctly defined when I use the default memory adapter.
I created a sailsjs app:
sails new testapp
sails generate api test testSet testGet
Installed connect-redis v1.4.7:
npm install connect-redis#~1.4.7
Set the configuration in config/session.js:
adapter: 'redis',
host: 'hoki.redistogo.com',
port: 10015,
db: 'redistogo',
pass: '88819aa089d3dd86235f9fad4cb92e48'
Set the configuration in config/socket.js:
adapter: 'redis',
host: 'hoki.redistogo.com',
port: 10015,
db: 'redistogo',
pass: '88819aa089d3dd86235f9fad4cb92e48'
Created some controller actions which get and set a session value:
UserController.js
testSet: function (req, res) {
req.session.testVar = "I am the test var!";
return res.ok();
},
testGet: function (req, res) {
return res.json({
testVar: req.session.testVar
});
}
And finally deployed to Heroku:
git init
git add .
git commit -m "Initial commit"
heroku create
heroku addons:add redistogo
git push heroku master
git
This is the error:
error: Sending 500 ("Server Error") response:
TypeError: Cannot set property 'testVar' of undefined
at module.exports.testSet (/app/api/controllers/TestController.js:46:25)
It seems like this simple example should work.
Here is a repo of the example above:
https://github.com/derekbasch/sailsjs-redistogo-testapp
Does anyone know what I am doing wrong?
UPDATE:
I tried using the MemoryStore adapter on Heroku to get/set a session variable. That failed with undefined also. It works locally. Now I am even more confused.
We are using rediscloud (sails app on heroku) and the db property is set to 0. Could this be the problem?
Also, you should parse the URL provided by heroku via en env variables. This is what we use (coffeescript):
parseRedisUrl = ( url ) ->
parsed = require( 'url' ).parse( url )
password = (parsed.auth || '').split( ':' )[1]
hostname: parsed.hostname
port: parsed.port
password: password
redis = parseRedisUrl( process.env.REDISCLOUD_URL || "redis://localhost:6379" )
module.exports.session =
secret: '...'
adapter: 'redis'
host: redis.hostname
port: redis.port
pass: redis.password
db: 0
ttl: 60 * 60 * 24

Resources