We are using Redis 3.2 (64 bit) (https://github.com/MSOpenTech/redis/releases) on Windows Server 2012 R2 Standard, for in-memory data caching.
Have been able to write redis-server logs by setting the logfile parameter in redis.conf file but, could not specify log file max size & subsequent rollover.
Would want to know whether there is a way to specify log file rotation in Redis conf / probably pass that as a parameter when starting the Redis server daemon process.
It would really help to get any suggestions on this front.
Thanks & regards,
Surjit
Although this is an old question... MS Open Tech Redis project has been abandoned.
You can checkout Memurai, which derives from it link.
Disclaimer: I work in Memurai.
Related
I am using redis for pub/sub as well as for server side cache. I mean my app server has redis server running as one process (functioning as a cache as well) . I have several thin clients (running redis client) connected to this app server in pub/sub mode. I would like to know where redis stores the cache data ? in server alone or there will be a copy in the clients as well. Also is it a good idea to use Redis in this fashion if there are close to 100 redis clients connected to server through pub/sub channel.
Thanks
Redis is a (sort of) in-memory noSQL database; but I found that my copy (running on linux) dumps to /var/lib/redis/dump.rdb
Redis can manage really big numbers of connections, by default its in-memory store (thanks to storing stuff in RAM it can be so fast).
But in the same time it can be configured as a persistent store, so dumping cached data (every x time or every x updated keys) to disk.
So it can be configured depending on your needs, have a look here.
All the cache data will be stored in the memory of the server provided to the config of running redis server.
The clients do not hold any data, they only access the data stored by the redis server.
I just installed redis on mac via homebrew. Without any configuration, I
found the dump.rdb is in my working directory (where I launched redis-server).
You can figure that out with the config command.
redis-cli config get dir
However as far as I know pub/sub data is volatile and not stored nor cached in redis at all. If you need that, you should look for a dedicated message broker like for example RabbitMQ.
On my Ubuntu, it was at /var/lib/redis/dump.rdb. On my macOS (installed via brew), it was at /usr/local/var/db/redis/dump.rdb.
Default location
/var/lib/redis/
Redis save all data in memory of server and rarely save date to disk.
For server<>client flow - all data transport with server.
Redis can processing number of clients ... default limit - 10.000
If you need less .. you must reconfigure OS, Server Settings etc. - http://redis.io/topics/clients
As I understood about the question your concern is about the Radis server memory and the client (application) memory.
I would like to know where redis stores the cache data ? in server alone or there will be a copy in the clients as well.
The Redis 6's client-side caching is what you actually looking for. There server and application both stores copies an keep in sync through a protocol communication. Eventhough they have implemented few ways to accomplish it following example (picked from the docs) mechanism will help you to understand it well.
Client 1 -> Server: CLIENT TRACKING ON
Client 1 -> Server: GET foo
(The server remembers that Client 1 may have the key "foo" cached)
(Client 1 may remember the value of "foo" inside its local memory)
Client 2 -> Server: SET foo SomeOtherValue
Server -> Client 1: INVALIDATE "foo"
Hope this helps. See that nice docs for more elaboration.
I read on MOS Doc ID 1945619.1 that starting with the 12.1.3 Oracle HTTP Server (OHS), the mod_plsql feature has been deprecated and will not be included with the 12.2 Oracle HTTP Server.
For the future, Oracle recommends moving to Oracle REST Data Services (formerly known as Oracle APEX Listener) as an alternative to mod_plsql.
Our shop have a lot of mod_plsql applications (i.e. applications written usinjg HTP/HTF packages) in production. Since I don't know anything about Oracle REST Data Services I'm asking you if we can migrate the old applications to this new product without changing the code.
Thank you.
Kind regards, Cristian
Doug McMahon (Oracle employee) has a great open source module for Apache.
Apache PL/SQL Gateway Module
(mod_owa)
https://oss.oracle.com/projects/mod_owa/dist/documentation/modowa.htm
I am using it in a production environment and I highly recommend it. It's really fast and rock solid.
You need to do some compiling but it's worth it being able to use Apache 2.4 and mod_plsql.
Steps:
download httpd 2.4.? from apache.org + extract
If Centos 6 or less download apr and apr-util
configure with enable-so, make and make install
./configure --enable-so --with-apr=/usr/local/apr --with-apr-util=/usr/local/apr
Download mod_owa + unzip
create empty directory. Copy all files from "apache24" into new folder. Copy all files from "src" to new folder
enter new folder and edit modowa.mk <-- important add $ORACLE_HOME, edit APACHE_TOP
Copy mod_owa.so to apache's modules
Add a modowa.conf in Apache's conf/ dir.
Example modowa.conf:
loadModule owa_module modules/mod_owa.so
<Location /pls>
Options None
SetHandler owa_handler
OwaUserid user/pass
OwaNLS AMERICAN_AMERICA.AL32UTF8
OwaPool 100
OwaStart "package.procedure"
OwaDocProc "wwv_flow_file_mgr.process_download"
OwaDocTable photos_upload BLOB_CONTENT
OwaUploadMax 50M
OwaCharset "utf8"
order deny,allow
allow from all
OwaReset LAZY
OwaCharsize 4
OwaFlex package.procedure
OwaHttp REST
</Location>
Before starting httpd ORACLE_HOME, NLS_LANG needs to be set (ORACLE_SID also if local). It needs access to an Oracle Home with libclntsh.so. (Oracle client will do).
I simply added oracle.conf (one line full path to oracle home/lib) under /etc/ld.so.conf.d (+ ldconfig)
Really scalable and a much cleaner setup then OHS.
My shop is pretty much in the same situation as you are.
We also have some very large mod_plsql/htp based applications and will have to migrate to the Oracle REST Data Services at some point.
We have already spend quite some time in testing different ORDS configuration and our overall conclusions are:
only APEX applications are fully supported
key functionality is still available
harder to configure and maintain
slight performance degradation
some mod_plsql configuration options do no longer exist or have changed
The biggest problems we are currently facing (and actually preventing us from switching to ORDS) are some restrictions when using non-APEX (pure HTF/HTP) applications.
We already filed some SR's because some functionality in ORDS (for example the file upload and download API) is only available when running an APEX application.
The biggest hurdle to get over is setting up Oracle Rest Services (ORS) and securing it. Once this is done, your web toolkit apps will work the same. The url may slightly change, so if you've referenced URLs using full paths as opposed to relative paths you might need modify code.
I am not sure if ORS is as powerful as Apache in areas like mod_rewrite, mod_proxy, virtual hosts with multiple ip addresses, etc...
Another open source alternative is tox.
After following this simple tutorial http://www.louisaslett.com/RStudio_AMI/ and video guide http://www.louisaslett.com/RStudio_AMI/video_guide.html I have setup an RStudio environment on EC2.
The only problem is, I can't upload large files (> 1GB).
I can upload small files just fine.
When I try to upload a file via RStudio, it gives me the following error:
Unexpected empty response from server
Does anyone know how I can upload these large files for use in RStudio? This is the whole reason I am using EC2 in the first place (to work with big data).
Ok so I had the same problem myself and it was incredibly frustrating, but eventually I realised what was going on here. The default home directory size for AWS is less than 8-10GB regardless of the size of your instance. As this as trying to upload to home then there was not enough room. An experienced linux user would not have fallen into this trap, but hopefully any other windows users new to this who come across this problem will see this. If you upload into a different drive on the instance then this can be solved. As the Louis Aslett Rstudio AMI is based in this 8-10GB space then you will have to set your working directory outside this, the home directory. Not intuitively apparent from Rstudio server interface. Whilst this is an advanced forum and this is a rookie error I am hoping no one deletes this question as I spent months on this and I think someone else will too. I hope this makes sense to you?
Don't you have shell access to your Amazon server? Don't rely on RStudio's upload (which may have a 2Gb limit, reasonably) and use proper unix dev tools:
rsync -avz myHugeFile.dat amazonusername#my.amazon.host.ip:
on your local PC command line (install cygwin or other unixy compatibility system) will transfer your huge file to your amazon server, and if interrupted will resume from that point, will compress the data for transfer too.
For a windows gui on something like this, WinSCP was what we used to do in the bad old days before Linux.
This could have something to do with your web server. Are you using nginx or apache as your web server. If so you can modify the upload feature in your nginx server. If you are running nginx on the front end of the web server I would recommend the following fix in your nginx.conf file.
http {
...
client_max_body_size 100M;
}
https://www.tecmint.com/limit-file-upload-size-in-nginx/
I had a similar problems with a 5GB file. What worked for me was to use SQLite to create a database with the csv file that I needed. Use SQLite code to bring create the database. Then I used a function in RStudio to communicate with the local database. In that way, I was able to bring in the csv file. I can track down the R code that I used if you like.
I'm trying to keep a file updated real time with the server. Its more like a real time syncing which has a very small delay. Is there any application that lets me do this? Or would you suggest me using a local host as a server?
I dont know how you are connected to your server - but i assume this will be something like SCP / SFTP / FTP and i dont know your OS. WinSCP will do excatly this what you need, you can set it to watch your Filesystem (to a specified folder) and it will update the server files as soon as your file on your drive changes.
It also supports command line features so that you can use it within your own applications.
I always forget to check what's going on in IIS on our webservers, and am wondering: is there some stupid applet or something that always runs locally that I can click on to check event logs and IIS logs on a remote machine?
Mark
You can set up samurize to follow the output of the logging on the local and remote machines but it requires some setup.
You can use a remote shell utility such as OpenSSH to connect to remote machines securely.
One at a time. Compmgmt.msc -> connect to another computer.
But one at a time is boring. Monitoring dozens of machines? I've been using logparser from MS for my log monitoring needs. I run a query that dumps errors and warnings to a csv file a few times a day.
So far, I've only used it to aggregate event logs across the dozen servers in our QA environment, but it appears to take many forms on log input, including IIS. A pseudo log file query (don't have samples with me)
logparser "Select [eventtype], [message] into output.csv FROM \\server1\system, \\server2\system" -i EVT
This shows: You can aggregate multiple servers. You tell it the input format - it supports a dozen log types. You can dump it into a csv file. It looks sort of like SQL. This article in security focus has an IIS log sample.
I'm not an applet type of guy, so I haven't though much about desktop widgets to do this.