I'm a little stuck in a postinst file for Ubuntu.
The problem is when upgrading the package. I have some sqlite databases (marked as config files to save them from upgrades) in which I want to modify in a specific version (add some columns). As only I want to modify the database when installing an specific version (for example from version 3 or older to 4) I check if $2 is not null.
The approach to make the database upgrading is as follows:
first I make a backup of the databases
make new table
Then I alter the tables
copy the rows from the old table to the new one
The problem is if something goes wrong, the database already will be modified but the package will be at the new version (the four) with config version at 3. If I want to try install again the package in order to get the package ok, the postinst will fail because the database was already modified.
One thing that came to my mind was unset -e, but I find this very unpleasant.
I've searched in the documentation about how to revert fails from postinst, such as if it's called another script with different arguments, but in the documentation Debian says nothing useful.
Maybe the postinst script is not the best place to modify databases?
Thanks
Best regards
One thing you could do is check if the field already exists on the table, and alter in that case.
That is not dependent on if this is an upgrade or not.
Like this for example:
if sqlite3 /collection.db '.schema media' | grep -q 'new_field text' ; then
echo field already exists, continuing
else
sqlite3 /collection.db 'alter table media add column new_field text'
fi
Also, I would recommend the database schema to be manipulated by the application, and not dpkg.
Imagine that the user might restore a backup from an older version and "break" your software.
It should be clear to the user that the database of older versions is incompatible with the newer version.
I would recommend you to search the internet for help in creating a "migration plan".
Basically that means that you keep track of what "version" the database is and apply sequentially the database modifications when needed, in a way that is very similar to how a patch works.
Related
We are working on Joomla3.9 project, have different environments and are using git as vcs. So every developer works on is own branch. It would be nice to have a database compare function like in TYPO3 or Contao (see the database differences after updating the project and apply the database changes just by one click). Or like the laravel migration system.
Any developer should easily update his own lokal database after database changes where made due an extension update via backend or by another developer. And of course the staging or live system must be updated easily too. We don't want to execute sql-scripts with the changes in phpMyAdmin.
We have tried https://dbv.vizuina.com/ . This is not the 100% solution. Like there is no cli support to start the migration process by an update script on the server.
Does anyone have a solution or knows an extension that can solve this problem? Or can this be handled with core Joomla functions (maybe with a little adjustment)?
So far, I've seen three possibilities to execute modifications to one ore many extension tables
1: Use the extension - revision control in the schema table. So add a new sql-file with an increased version number compared to the version number in the schema-table for this extension. Increase also the version in the manifest.xml and zip the extension again.
Reinstall the extension via extension->manage->install. So the new sql-file with the increased version number will be executed.
2: like the point above, but install the extension via joomla update mechanism (update server).
3.: create a new sql-file in sql/folder of the extension. No version name is needed for the new file, just update.sql oder another filename. Execute this script in script.php in update()-method, after the extension is installed (in this case it's an update) again.
The third possibility might be interesting. It should be possible to trigger the update()-method with a cli command / function, so that the method can be triggered via a script on the server.
But how can I get the info, which update-scripts have already been executed? Let's say I have 3 update files in sql-folder. update-1.sql, update-2.sql and update-3.sql.
update-1.sql has already been executed. So I don't want to execute this sql-file again - only the other two.
The schema-table is only used with the first two options. Do I have the info somewhere or must I manage the infos which update-scripts have been executed myself?
The answer related to versioning database for extensions depends on whether these extensions are tightly coupled to the application or need to be reusable to other applications also.
The latter case normally means that each extension accesses its own custom tables, in which case you should keep separate versioning for the database than for the extensions.
App version history can be kept in a db_version table. Then an insert statement is added at the end of each update script (adding an incremental version number). e.g.
insert into db_version(version,author,description) values(003,'Verna.Collins', 'removing obsolete column');
Provided that you need to apply data migration on extensions also, you need to maintain a db_version_extensions table which keeps version history for each of the extensions separately. e.g.
'001' 'extension1','Mandy.Aguilar','initial version'
'002' 'extension1','Mandy.Aguilar','adding extra column'
'001' 'extension2','Edna.Potter','initial version'
'002' 'extension2','Elvira.Townsend','dropping unused table'
..etc
Each extension zip should keep initial creation script and all sql-update files(which should normally not interfere with the rest of the app tables).
After pull it will be relatively easy to execute all the scripts with filename version greater than the last version number written in the database. This should be done for the app and for each extension separately.
Now if the extensions are tightly coupled to the app, it means that they might be using/updating tables of the app. For extensions of this type, you can add the updates as part of the application updates. These extensions could even be developed at the same repo, and be kept as directories instead of zip files.
Not sure if joomla supports any tools for automating the process of performing incremental db updates, but a nice tool is flyway, with ports for command-line, maven and graddle. See: how does flyway work
I have developed a Joomla extension and in the newest version the need to delete one column from a table belonging to my plugin has arisen.
As far as I know there is no easy way to delete a column in Mysql only if it exists. Therefore, the only possibility I see at the moment to achieve that, would be to use custom php upgrade code, as shown here.
But this problem would seem so common to me, that I would suspect to be an easier solution?
Is there maybe an easy way (provided through Joomla) to execute a schema change only when updating from a particular extension version?
Check Managing Component Updates with Joomla; starting with Joomla 1.6 there is a defined way to execute specific SQL instructions for an upgrade from one version to another. Basically, for each version an SQL file has to be created (even if it's empty or only contains comments) in a specific file structure. The linked article explains the details.
I have a database that is created by the WIX installer using SqlScript and SqlString tags. Inside the database is a table "AppVersion" with a field "VersionNumber".
I can create the database just fine, but I'm working on the upgrade script now. How can I make my upgrade MSI check the value of the AppVersion.VersionNumber field as a condition of upgrading?
I see tags to check registry values, but nothing so far on how to check database values as a condition.
Thanks in advance.
EDIT Oh yeah, saving the value in the registry is not viable because the databases are backed up / restored to different machines often enough that the installer itself must use the database version field.
Generally I put all of that logic into SQL. A script should be smart enough to create a table only if it doesn't already exist and to alter a table only if it needs to. It's not the job of the installer to do all that logic and then decide which SQL scripts to run.
Whenever I run propel-gen on a clean database (no tables defined), it generates errors because it first tries to DROP a table (Which doesnt currently exist) before creating it.
Is there any properties I can change to fix this?
So far I've found this:
http://www.propelorm.org/ticket/732
But it would be nice to do something like "DROP TABLE IF EXISTS name"
I faced the same problem some months ago but couldn't find more than the trac ticket (you will find, as well, that I have commented it :-)).
Maybe try to apply Elan Ruusamäe's patch if you find this bug very annoying...
If I have enough time/motivation, I may write a patch too (to make Propel issue a CREATE TABLE IF NOT EXISTS for DBMS supporting it) and hopefully the fix will be scheduled for one of the next versions.
I recently moved my whole local web development area over to using MacPorts stuff, rather than using MAMP on my Mac. I've been getting into Python/Django and didn't really need MAMP any more.
Thing is, I have uninstalled MAMP from the Applications folder, with the preferences file too, but how come when I run the 'locate MAMP' command in the Terminal it still shows all my /Applications/MAMP/ stuff as if it's all still there? And when I 'cd' into /Applications/MAMP/ it doesn't exist?
Something to do with locate being a kind of index searching system, hence things these old filepaths are cached? Please explain why, and how to sort it so they don't show anymore.
You've got the right idea: locate uses a database called 'locatedb'. It's normally updated by system cron jobs (not sure which on OS X); you can force an update with the updatedb command. See http://linux-sxs.org/utilities/updatedb.html among others.
Also, if you don't find files which you expect to, note this important caveat from the BUGS section of OSX' locate(1) man-page:
The locate database is typically built by user ''nobody'' and the
locate.updatedb(8) utility skips directories which are not readable
for user ''nobody'', group ''nobody'', or world. For example, if your
HOME directory is not world-readable, none of your files are in the database.
The other answers are correct about needing to update the locate database. I've got this alias to update my locate DB:
alias update_locate='sudo /usr/libexec/locate.updatedb'
I actually don't use locate all that much anymore now that I've found mdfind. It uses the spotlight file index which OSX is much better at keeping up to date compared to the locatedb. It also has quite a bit more power in what it can search from the command line.
Indeed the locate command searches through an index, that's why it's pretty fast.
The index is generated by the updatedb command, which is usually run as a nightly
or weekly job.
So to update it manually, just run updatedb.
According to the man page, its database is updated once a week:
NAME
locate.updatedb -- update locate database
SYNOPSIS
/usr/libexec/locate.updatedb
DESCRIPTION
The locate.updatedb utility updates the database used by locate(1). It is typically run once a week by
the /etc/periodic/weekly/310.locate script.
Take a look at the locate man page
http://unixhelp.ed.ac.uk/CGI/man-cgi?locate+1
You'll see that locate searches a database, not your actual filesystem.
You can update that database by using the updatedb command.
Also, since it's a database, unless you do update it regularly, locate wouln't find files that are in your filesystem that arn't in the database.