I'm running Magento EE 1.11 and I've deployed SOLR 4.1 with tomcat7. I've copied solrconfig.xml and schema.xml provided by Magento and fixed all the issues SOLR was having with the two files because they were intended for SOLR 3.6 and properly configured Magento.
Now, when I reindex search in Magneto at the very last moment SOLR spits out this exception:
org.apache.solr.common.SolrException: Unknown commit parameter 'waitFlush'
and Magento reverts back to saying a search re-indexing is required. Has anyone ran into this problem? From all the googling I've done there seems to be a patch for this, but where and how do I apply it?
You should really use SOLR 3.x rather than 4 with Magento EE.
Here's a walk-through for configuration and installation.
On Debian/Ubuntu
The most straightforward installation is pretty easy using tomcat and your package manager. The dependencies will be met automatically.
apt-get install tomcat6
On CentOS/RedHat
You need to grab some alternative repo's to make this possible
Eg.
rpm -Uvh http://download.fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm
rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
Then you can install the package from yum
yum install yum-priorities ant tomcat6 tomcat6-admin
cd /usr/src/
mkdir sun-java
cd sun-java
Now it gets a little trickier. Sun used to permit direct downloads; but they now have a stupid session validation in place - so download the binary via your PC and upload it to the machine.
You need both the Linux JDK and JRE.
The commands would have been:
wget -O jdk.rpm.bin http://download.oracle.com/otn-pub/java/jdk/6u29-b11/jdk-6u29-linux-x64-rpm.bin
wget -O jre.rpm.bin http://download.oracle.com/otn-pub/java/jdk/6u29-b11/jre-6u29-linux-x64-rpm.bin
You can alternatively use OpenJDK
wget http://jpackage.org/jpackage50.repo -O /etc/yum.repos.d/jpackage50.repo
yum install -y java-1.6.0-openjdk
Once you've uploaded the binaries
chmod +x *.bin
./jre.rpm.bin
./jdk.rpm.bin
ln -s /var/lib/tomcat6 /usr/share/tomcat6
Then the remaining steps
Then drop in your respective selection of solr
mkdir /usr/src/solr
cd /usr/src/solr
wget http://mirrors.ukfast.co.uk/sites/ftp.apache.org/lucene/solr/3.6.1/apache-solr-3.6.1.tgz
tar xvfz apache-solr-3.6.1.tgz
cd apache-solr-3.6.1
cp dist/apache-solr-*.war /var/lib/tomcat6/webapps/solr.war
mkdir -p /var/lib/tomcat6/solr
Then add the Magento solr configuration
INSTALL_DIR="/var/lib/tomcat6/solr"
touch $INSTALL_DIR/solr.xml
CORES=( "staging" "development" "live" )
for CORE in "${CORES[#]}"; do
mkdir -p $INSTALL_DIR/$CORE/conf $INSTALL_DIR/$CORE/data
cp -par /usr/src/solr/apache-solr-3.6.1/example/solr/conf/* $INSTALL_DIR/$CORE/conf
cp -par /home/path/public_html/lib/Apache/Solr/Conf/* $INSTALL_DIR/$CORE/conf
done
Then set up the cores
cat > /var/lib/tomcat6/solr/solr.xml << EOF
<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="true" sharedLib="lib">
<cores adminPath="/admin/cores">
<core name="staging" instanceDir="staging" config="solrconfig.xml" schema="schema.xml" />
<core name="development" instanceDir="development" config="solrconfig.xml" schema="schema.xml" />
<core name="live" instanceDir="live" config="solrconfig.xml" schema="schema.xml" />
</cores>
</solr>
EOF
Then finally, clean up permissions and restart solr
chown -R tomcat6:tomcat6 /var/lib/tomcat6/solr
/etc/init.d/tomcat6 restart
Then in Magento, you've now got 3 possible independent cores you can use for your store environments.
staging/solr
development/solr
live/solr
Attribution: http://www.sonassi.com/knowledge-base/multiple-solr-cores-for-magento-on-debianubuntucentosredhat/
Related
Noob alert... I am fairly new to both Linux and Apache/SVN, so this issue below is likely a stupid user error (stupid referring to both user and error)
Usually I can google my way to a solution, but I have been stumped for a few days on the "conical" error below. There are not many google references to "svn_uri_is_canonical(child_uri, NULL)" that don't also include accessing "error: git-svn died of signal 6" which I did not see
Any advise as to where I should be looking to solve issue below would be greatly appreciated
When I attempt to check out the svn repository that was created during the SVN install I see the following error
$ svn checkout http://127.0.0.1/svn/first-repo
Redirecting to URL 'http://127.0.0.1/svn/first-repo/':
svn: /build/subversion-Z4OiCa/subversion-1.13.0/subversion/libsvn_subr/dirent_uri.c:1562: uri_skip_ancestor: Assertion `svn_uri_is_canonical(child_uri, NULL)' failed.
Aborted (core dumped)
Some details about installation
From my Browser, entering "http://127.0.0.1/svn/first-repo/" displays:
first-repo - Revision 0: /
--------------------------------------------------------
Powered by Apache Subversion version 1.13.0 (r1867053).
Running svnadmin to check the repo:
$ svnadmin verify /var/www/svn/first-repo
* Verifying metadata at revision 0 ...
* Verified revision 0.
From dav_svn.conf (only changes made)
Alias /svn /var/www/svn
<Location /svn>
DAV svn
SVNParentPath /var/www/svn
AuthType Basic
AuthName "Subversion Repository"
AuthUserFile /etc/svnusers
Require valid-user
</Location>
contents of "os-release":
NAME="Linux Mint"
VERSION="20.2 (Uma)"
ID=linuxmint
ID_LIKE=ubuntu
PRETTY_NAME="Linux Mint 20.2"
VERSION_ID="20.2"
HOME_URL="https://www.linuxmint.com/"
SUPPORT_URL="https://forums.linuxmint.com/"
BUG_REPORT_URL="http://linuxmint-troubleshooting-guide.readthedocs.io/en/latest/"
PRIVACY_POLICY_URL="https://www.linuxmint.com/"
VERSION_CODENAME=uma
UBUNTU_CODENAME=focal
Apache version:
$ apachectl -v
Server version: Apache/2.4.41 (Ubuntu)
Server built: 2021-10-14T16:24:43
SVN version:
svnadmin --version
svnadmin, version 1.13.0 (r1867053)
compiled Mar 24 2020, 12:33:36 on x86_64-pc-linux-gnu
For installation, i followed this guide
https://linuxtechlab.com/simple-guide-to-install-svn-on-linux-apache-subversion/.
I ran into a problem with executing the command:
$ sudo apt-get install apache2 libapache2-mod-svn libapache2-svn
E: Unable to locate package libapache2-svn
Based on one google result, that package was referred to a "dummy" & not required... not sure if it is related to this issue
I uninstalled and tried again from this guide (very similar instructions with minor differences)
https://www.ubuntupit.com/how-to-install-and-configure-apache-svn-server-on-linux-desktop/
I again ran into a problem with executing the command:
$sudo apt-get install subversion libapache2-mod-svn libapache2-svn libsvn-dev
E: Unable to locate package libapache2-svn
the same conical error still occurs when I attempt checkout
Thanks
Read the doc CAREFULLY!!!
Who, when, where told you to use for the same svn-PATH both Location and Alias and even worse, with the same name?!
In assumption, that all prerequisites are met, you have
Start with minimal Location section without preceding Alias, don't mix real pre-existing physical path inside web-server with somehow virtual location (existing /var/www/svn in case of server root in /var/www/ and /svn location is very big mistake), use smth. like (move repo accordingly)
<Location /svn>
DAV svn
SVNParentPath /var/repos
</Location>
Test functioning of your (still) dummy system with existing repo (better not as 127.0.0.1, but with defined in apache's conf ServerName) http://127.0.0.1/svn/first-repo/not with web-browser or git-svn bridge, but with plain pure CLI svn client
After successful checkout of "open" repo, expand configuration by auth-option and re-check both (checkout and commit) basic operations
PS: It's extremely The Bad Idea (tm) to physically place repositories inside ordinary web-space, while not prohibited directly it's at least totally insecure
I have a Python script with a Windows .exe dependency, which in return relies on a (closed-source) Windows DLL. The Python script runs just fine in Ubuntu via a call to Wine.
Is it possible (and practical) to run this on AWS Lambda?
What would be involved in preparing the code package?
Update: the lambda container image feature supports images up to 10gb. I haven't tried it but I think that would be a viable approach, and wouldn't require the hacks I did below to reduce the wine build size.
TL;DR;
Is it Possible? Yes.
Is it practical? The approach I tried is not. A better approach might be to try and put wine into different lambda layers or a custom execution environment.
Will it work for you? It depends, deployment package size and disk space are the limiting factors.
Old, somewhat hacky method to fit wine into the regular lambda environment:
I compiled a custom wine with minimal dependencies for lambda, compressed it and then put it onto S3.
Then, in the lambda at runtime, I downloaded the archive, extracted it to /tmp and ran it with a custom empty wine prefix.
My test windows executable was 64bit curl.exe.
1. Compile Wine for Lambda
From https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html, I first tried amzn-ami-hvm-2018.03.0.20181129-x86_64-gp2, but it had an older compilation environment and wouldn't configure.
With AMI amzn2-ami-hvm-2.0.20190313-x86_64-gp2 on a t3.2xlarge ec2, I was able to configure and compile. These are the commands I used, references aws-compile and building-wine:
> sudo yum groupinstall "Development Tools"
> mkdir -p ~/wine-dirs/wine-source
> git clone git://source.winehq.org/git/wine.git ~/wine-dirs/wine-source
> cd ~/wine-dirs/wine-source
> ./configure --enable-win64 --without-x --without-freetype --prefix /opt/wine
> make -j8
> sudo mkdir -p /opt/wine
> sudo chown ec2-user.ec2-user /opt/wine/
> make install
> cd /opt/
> tar zcvf ~/wine-64.tar.gz wine/
This was only a 64-bit build. It also had almost no other optional wine dependencies.
2. Reduce the size of the Wine build further
I removed a lot of optional dependencies from the wine build at compilation time, but it was still too big. /tmp is limited to 500MB.
I deleted files in the package subdirectories, including what looked like optional libs, until I got it down to around 300MB uncompressed.
I verified that wine would still run curl.exe after deleting files from the build.
3. Compress it
I created a tar.bz2 of wine and curl with default bz2 options, it ended up around 80MB. The compressed and extracted files together required about 390MB.
That way there is enough room to both download the archive and extract it to /tmp inside the lambda.
> du -h .
290M ./wine/lib64/wine
292M ./wine/lib64
276K ./wine/share/wine
8.0K ./wine/share/applications
288K ./wine/share
5.0M ./wine/curl-7.66.0-win64-mingw/bin
5.0M ./wine/curl-7.66.0-win64-mingw
12M ./wine/bin
308M ./wine
390M .
> ls
wine wine.tar.bz2
4. Upload wine.tar.bz2 to S3
Create an S3 bucket and upload the wine.tar.bz2 file to it.
5. Create the Lambda
Create an AWS Lambda using the python 3.7 runtime. While this uses a different underlying AMI than what wine was built on above, it still worked.
In the lambda execution role, grant access to the S3 bucket.
RAM: 1024MB. I chose this because lambda CPU power scales with the memory.
Timeout: 1 min
6. Lambda code:
I needed to follow the advice from this question and answer to change the wine prefix inside the lambda. I also turned off the display as it suggested.
e.g.:
handler():
... download from S3 to /tmp, cd to /tmp
subprocess.call(["tar", "-jxvf", "./wine.tar.bz2"])
os.environ['DISPLAY'] = ''
os.environ['WINEARCH'] = 'win64'
os.environ['WINEPREFIX'] = '/tmp/wineprefix'
subprocess.call(["./wine/bin/wine64", "./wine/curl-7.66.0-win64-mingw/bin/curl.exe", "http://www.stackoverflow.com"])
Success!
I created openshift project with DIY cartridge, added postresql for DB.. now I pushed sources to cloud, but I'm getting error like the guy here (but he was not using springboot): I can't start tomcat 7 server on linux openshift - Failed to start end point associated with ProtocolHandler ["http-nio-12345"]
Failed to start end point associated with ProtocolHandler
It's obvious that if I run the app twice the error message 'Failed to start end point associated with ProtocolHandler' appears. I tried rhc app-tidy, restarted APP via openshift interface, i made change, pushed to cloud, build success, but the message appeared again. How to stop the app so that I can run it properly? Didn't the app being stopped after the restart I performed?
UPDATE: Have I chosen the right steps? (with DIY cartridge), do You have a simple (& working) guide how to deploy such an app to openshift? I tried few I found over the net, but none of them worked :(
Here is steps I follow using DIY cartridge, hope this would help
Go to home directory of project perform following tasks
Create Settings.xml and refer maven repo by updating settings.xml as shown in following xml snippet below. And also add the file to git repo
<settings>
<localRepository>${OPENSHIFT_DATA_DIR}/m2/repository</localRepository>
</settings>
Got to .openshift directory and delete all the files except action_hooks
Go to action_hooks and perform following task
i. Delete README.md
ii. Create deploy file (note the file extension while create should not be txt) and add following code snippet
(It basically creates and downloads and sets up java8 and maven
#!/bin/bash
set -x
if [ ! -d $OPENSHIFT_DATA_DIR/m2/repository ]
then
cd $OPENSHIFT_DATA_DIR
mkdir m2/repository
fi
if [ ! -d $OPENSHIFT_DATA_DIR/jdk1.8.0_51 ]
then
cd $OPENSHIFT_DATA_DIR
wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u77-b03/jdk-8u77-linux-x64.tar.gz
tar xvf *.tar.gz
rm -f *.tar.gz
fi
if [ ! -d $OPENSHIFT_DATA_DIR/apache-maven-3.3.3 ]
then
cd $OPENSHIFT_DATA_DIR
wget http://mirror.fibergrid.in/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
tar xvf *.tar.gz
rm -f *.tar.gz
fi
cd $OPENSHIFT_REPO_DIR
export M2=$OPENSHIFT_DATA_DIR/apache-maven-3.3.3/bin
export MAVEN_OPTS="-Xms500m -Xmx500m"
export JAVA_HOME=$OPENSHIFT_DATA_DIR/jdk1.8.0_51
export PATH=$JAVA_HOME/bin:$M2:$PATH
mvn -s settings.xml clean install
iii. Add the deploy file to git
iv. Edit start file with following snippet.
#!/bin/bash
source $OPENSHIFT_CARTRIDGE_SDK_BASH
set -x
export JAVA_HOME=$OPENSHIFT_DATA_DIR/jdk1.8.0_51
export PATH=$JAVA_HOME/bin:$PATH
cd $OPENSHIFT_REPO_DIR
nohup java –Xms500m –Xmx500m -jar target/*.jar --server.port=${OPENSHIFT_DIY_PORT} --server.address=${OPENSHIFT_DIY_IP} &
v. Edit stop with following code snippet
#!/bin/bash
source $OPENSHIFT_CARTRIDGE_SDK_BASH
PID=$(ps -ef | grep java.*\.jar | grep -v grep | awk '{ print $2 }')
if [ -z "$PID" ]
then
client_result "Application is already stopped"
else
kill $PID
fi
vi. Make deploy file executable (Note: This is very important and its common mistake, if not executable your project will not start)
Commit the changes along with your maven src and pom.xml your application should automatically start once the commit is done, Note: for first time it will take time as it needs to download maven repo.
For debuging purpose, its better to add logback.xml in your class path(src/main/resources)
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="LOG_FILE" value="${OPENSHIFT_LOG_DIR}/kp-apps.log" />
<include resource="org/springframework/boot/logging/logback/base.xml" />
<logger name="org.springframework.web" level="DEBUG" />
Just started using EC2, launched the instance with Amazon Linux AMI, installed my web app on it...and I thought I was ready to go.
I go to the public DNS they gave me for my instance and nothing happens. I get the Google Chrome "Oops" ...
After re-reading the doc I saw some notes that I need to launch the web server. By doing this:
sudo chkconfig httpd on
sudo service httpd start
But I can't seem to locate these files.
Any ideas what I am doing wrong or where those files are located? The closest I got was /etc but even if I try the command lines from there, I get the same errors.
Thanks for your help.
UPDATE
Here is an image of the security group I have for my instance. This is correct right?
Is it possible you don't have Apache/Tomcat installed? Check with:
rpm -q httpd
rpm -q tomcat5
If they are not installed, run:
sudo yum install httpd tomcat5
Which type of webapps(php/java/Ruby etc) you are trying to deploy? If it is php, where you are trying to put in apache directory? For example default root document location in apache is /var/www/html . If you put your webapp inside /var/www/html it should be available through direct url http://x.x.x.x/{nameofwebapp}
Background:
I recently installed MAMP, and am using it as a production server. The server setup did not come with an FTP server, and from what I've read, you can set up an FTP server via mod_ftp, an Apache module. I am not an expert with Apache software or server admin, although I can learn quickly. I can get to the following point and then I get stuck. Can someone please help me out?
I checked out the mod_ftp module files from the repository, here:
http://svn.apache.org/repos/asf/httpd/mod_ftp/trunk/
and I unzipped the contents into:
/Applications/MAMP/mod_ftp
I opened the README-FTP file (here):
http://svn.apache.org/repos/asf/httpd/mod_ftp/trunk/README-FTP
README-FTP:
To build and install as a DSO outside of the httpd source
build, from the ftp source root directory, simply;
./configure.apxs
make
make install
...
To build static, or as a DSO but within the same build as httpd,
copy the entire ftp source directory tree on top of your existing
httpd source tree, and from the httpd source root directory
./buildconf (to pick up ftp)
./configure --enable-ftp {your usual options}
and proceed as usual.
Some Questions:
"build and install a DSO outside of the httpd source build, from the ftp source root directory" -- is the ftp source root directory the mod_ftp folder that I created from the zipped files I checked out from the repository?
What does it mean "outside of the httpd source build"? -- is this the ServerRoot value I set in the httpd.conf as "/Applications/MAMP/Library" ?
Likewise, what does "within the same httpd build" mean -- what location is this referring to?
How do I know whether I want a static or DSO build?
What is the statement: "copy the entire ftp source directory tree on top of your existing
httpd source tree" actually asking me to do? (on top of?? As in, in the parent directory of the httpd source tree, or in the same directory?)
If you've made it this far, I'd like to commend you!
From this point, I chose the first option, and entered the commands seen in README-FTP into my Terminal.
Here's what my terminal looks like:
$ ./configure.apxs
Configuring mod_ftp for APXS in /usr/sbin/apxs
Detecting features
Finished, run 'make' to compile mod_ftp
Run 'make FTPPORT=8021 install' to install mod_ftp
(The default FTPPORT is 21 if not specified)
The manual pages ftp/index.html and mod/mod_ftp.html
will be installed to help get you started.
The conf/extra/ftpd.conf will be installed as an example
for you to work from. In your configuration file,
/private/etc/apache2/httpd.conf
uncomment the line '#Include conf/extra/ftpd.conf'
to activate this example mod_ftp configuration.
$ make
Making all in modules/ftp
$ sudo make install
Password:
Making install in modules/ftp
/usr/share/apr-1/build-1/libtool --silent --mode=install cp mod_ftp.la /usr/libexec/apache2/
Installing configuration files
for i in /private/etc/apache2/httpd.conf /private/etc/apache2/original/httpd.conf; do \
if test -f $i; then \
(awk -f /applications/mamp/library/mod_ftp/build/addloadexample.awk \
-v MODULE=ftp -v DSO=.so -v LIBPATH=libexec/apache2 \
-v EXAMPLECONF=/private/etc/apache2/extra/ftpd.conf \
< $i > $i.new && \
mv $i $i.bak && mv $i.new $i \
) || true; \
fi; \
done
Preserving existing FTP documents
Installing header files
Installing online manual
$
So what do I do from here?
I don't see mod_ftp.so anywhere, and I am particularly looking in this directory:
/Applications/MAMP/Library/modules (where all of Apache's other mod_*.so files are...)
and this directory:
/Applications/MAMP/mod_ftp/modules/ftp (where all of mod_ftp's various .c, .h and other files are)
Ultimately, I think the problem I am running into is that I don't understand how the file structures between my mod_ftp source folder and the httpd source folders need to be integrated in order to get the module running properly. Also, I don't know what I don't know, so there is probably one simple question to ask, but unfortunately I can't figure out how to ask it. Thank you for your help and patience!
Cheers!
P.S., yes, I have scoured the internet for hours.
I ended up scrapping MAMP and using the Mac's built in server. Through the System Preferences > Sharing menu, you can enable file sharing, which has an Options pane that allows you to "Share files and folders using FTP." I was able to obtain a static IP address through Comcast Business, and configured port forwarding on port 21 in my router to accept traffic. Then, I could use my FTP client to connect to my router with something like "123.456.789:21" as my host. Wasn't the best or most secure solution, but it worked, so take it with a grain of salt.
Right, I finally managed to install this on Ubuntu LTS 16.04.
First of all, you should install svn and the apxs functionality by running
sudo apt-get install subversion apache2-dev.
Then, cd to a convenient folder, and run svn co http://svn.apache.org/repos/asf/httpd/mod_ftp/trunk/. This downloads everything in the folder named /trunk. Then, cd into the /trunk folder of the downloaded repo.
Then, run the stated instructions
./configure.apxs --> does something in the subfolder to enable makefile to work
make --> this compiles the contents of the repo and changes things around.
make install --> you may want to run with the suggested flag. Essentially copies things to where it needs to be copied, and creates the necessary modules.
The suggested ./buildconf and ./configure should only be done if you are compiling apache2 with ftp at the same time. Since you should already have apache2 installed, this is not the option that you should be doing. Just stick with the first set of instructions, which are used to compile mod_ftp independently of apache2 and patch things in as needed.
At this point, the installation should technically work. However, you are not fully out of the woods yet. If you restart apache2 at this point, it should fail to start. If you run systemctl -xe, you will see that it is due to syntax errors in various places of the config files where someone forgot to prepend a forward slash, so rather than being given relative to root, the directories being specified end up being relative to /etc/apache2 instead. Fix those, using the line numbers as a guide. The omissions may be found in the apache2.conf file that specify the location of the mod_ftp module, and in the ftpd.conf file that specify the location of the error log.
You now need to mess around with apache2.conf and ftpd.conf (found in the /extra subfolder of the /etc/apache2 folder). Make sure that the lines
include /etc/apache2/extra/ftpd.conf
LoadModule ftp_module /usr/lib/apache2/modules/mod_ftp.so are present and uncommented.
The first basically tells the main apache2.conf file to include the configuration files for mod_ftp to help with partitioning your http and ftp configuration settings. The second just makes sure that the ftp module is loaded so that it can interpret the directives in the ftpd.conf file. Thus, you won't need to add the line "FTP on" or specify ports, as those are handled in ftpd.conf perfectly well.
You should now be good to go. Just note that for some weird reason if you set the document root in ftpd.conf to be the same as that in apache2.conf, apache2 will still run normally. The ftp server will work normally but the http server will not work. No idea why, but if you want to do that a simple workaround is to just do a symlink to the http document root and set that as the ftp document root.