I have created emp table to store employees information and their PP size photo.
this table has empno (number), emp_image_link(varchar2), .... etc fields.
empno is auto generated using a database trigger (max empno+1).
Image : I don't want to store images into the database since I believe it will cause problems in terms of size, performance and portability. So images should be in the file system at D:\images\
and images URL should be D:\images\empno.jpg, which means emp_image_link field will contain only the image link.
I have searched Google a lot about this, everyone is discussing about how to store into the database.
I did not find any information about how to store only the link instead of the image.
I am going to use Oracle Forms Developer 11gR2.
Can anyone give me an idea of how I can do that please.
Thank you in advance.
Murshed Khan
"i dont want to store images into the database since it will cause
problem in terms of size, performance and portability i believe. so
images should be in the file system"
Your points are not valid ones.
Size. Passport photos are pretty small, so unless you are storing pictures with extremely high pixel counts they won't take up a lot of disk. Either way they will consume comparable amounts of space in the database and on the OS.
Performance. The only possible concern would be the network traffic between the database server and the middle-tier server. This would be a function of size, so may or may not be a real issue. Using na OS file store would introduce a time delay while you retrieve the JPG for each record.
Portability. An all-in-the-database solution is more portable than what you're proposing. Nothing breaks like directory paths.
One thing you haven't considered but you really should is DML on the employee records. If the pictures are stored in the database they are committed in the same transaction as (hence consistent with) the rest of the data, they are backed-up at the same time and they are recoverable in the same window. None of which applies to an OS directory on a separate server.
"Storing in the file system ... I got the solution using BFILE "
BFILE is the mechanism for linking a database record with an OS file. So it is the appropriate solution for the problem as you define it. But the BFILE points to files on the database server, so you would lose the only possibly efficiency to be gained from not storing records in the database, the network traffic between the database and middle tier servers. BFILEs would not be backed up with the database or subject to any transactional consistency.
"empno is auto generated using a database trigger (max empno+1)"
Another bad idea. It doesn't scale and more importantly it doesn't work in a multi-user environment. Please use a sequence, they're designed for this task.
Related
I wrote some basic blog system, which based on spring boot.
I'm trying to figure out, how can I create posts with videos and images, without the need to editing everything using HTML.
Right now, I am saving my blog posts in DB as plain text.
Is it possible to create content combined with text, images and videos , and saving this "content" as one row in my DB-Table, without creating connections between different tables?
Many thanks in advance.
Images and Videos are heavy content and storing them in database could be a costly affair, until you are developing application for research purpose. Also querying it from database and serving it over the network can impact your application performance.
If you want to store it in a single row that can be done as well using database BLOB object. But i would suggest to have 2 different tables. One containing the BLOB object of Image and Videos and other is your usual table containing blog as text and primary key of of BLOB table.
If you want to take your solution to go live, better use image-videos hosting servers because of following factors
Saves your database cost
Ensures 24x7 availability
Application performance is faster as these are hosted independent of application
Videos can directly be iframed i.e. you do not need to query complete MBs of record and serve over network
A strict answer to your question, yes, you can use BLOBs to store the video/images in the database. Think about them as a column that contains bytes of video or image.
For school/cases where you have a very small amount of videos/images its probably OK. However if you're building a real application, then don't do it :)
Every DBA will raise a bunch of concerns "why do not use Blobs".
So more realistic approach would be to find some "file-system" like storage (but distributed) style S3 in AWS, hardrive on server if you're not on cloud, etc.
Then store that big image / video there and get an identifier (like path to it if we're talking about the harddrive) and store that identifier in the database along with the metadata that you're already store probably (like blogPostId, type of file, etc.)
Once the application become more "mature" - you can switch the "provider" - Grow as you go. There are even cloud storages designed especially for images (like Cloudinary).
I’m in the middle of trying to migrate a large amount of data into a oracle database from existing excel-files.
Due to the large amount of rows loaded (10 000 and more) every time, it is not possible to use SQL Developer for this tasks.
In every work-sheet there’s data that need to go into different tables, but at the same time keep the relations and not dropping any data.
As for now, I use one .CSV file for each table and mapping them together afterwards. This is thou combined with a great risk of adding the wrong FK and with that screw up the hole shit. And I don’t have the time, energy or will for clean ups even if it is my own mess…
My initial thought was if I could bulk transfer with sql loader using some kind of plsql-script in maybe an ctl-file (the used for mapping the properties) but it seems like I.m quite out in the bush with that one… (or am I…? )
The other thought was to create a simple program In c# and use fastMember and load the database that way. (But that means that I need to take the time to actually make the program, however small it is).
I can’t possible be the only one that have had this issue, but trying to us my notToElevatedNinjaGoogling-skills ends up with either using sql developer (witch is not an alternative) or the bulk copy thing from sql load (and where I need to map it all together afterwards).
Is there any alternative solutions for my problem or is the above solutions the one that I need to cope with?
Did you consider using CSV files as external tables? As they act as if they were ordinary Oracle tables, you can write (PL/)SQL against them, inserting data into different tables in the target schema. That might give you some more freedom & control over what you are doing.
Behind the scene, it is still SQL*Loader.
I want save uploaded images in a bytea column in my PostgreSQL database. I'm looking for advice on how to how to save images from Rails into a bytea column, preferably with examples.
I use Rails 3.1 with the "pg" driver to connect to PostgreSQL.
It's often not a good idea to store images in the database its self
See the discussions on is it better to store images in a BLOB or just the URL? and Files - in the database or not?. Be aware that those questions and their answers aren't about PostgreSQL specifically.
There are some PostgreSQL specific wrinkles to this. PostgreSQL doesn't have any facilities for incremental dumps*, so if you're using pg_dump backups you have to dump all that image data for every backup. Storage space and transfer time can be a concern, especially since you should be keeping several weeks' worth of backups, not just a single most recent backup.
If the images are large or numerous you might want to consider storing images in the file system unless you have a strong need for transactional, ACID-compliant access to them. Store file names in the database, or just establish a convention of file naming based on a useful key. That way you can do easy incremental backups of the image directory, managing it separately to the database proper.
If you store the images in the FS you can't easily† access them via the PostgreSQL database connection. OTOH you can serve them directly over HTTP directly from the file system much more efficiently than you could ever hope to when you have to query them from the DB first. In particular you can use sendfile() from rails if your images are on the FS, but not from a database.
If you really must store the images in the DB
... then it's conceptually the same as in .NET, but the exact details depend on the Pg driver you're using, which you didn't specify.
There are two ways to do it:
Store and retrieve bytea, as you asked about; and
Use the built-in large object support, which is often preferable to using bytea.
For small images where bytea is OK:
Read the image data from the client into a local variable
Insert that into the DB by passing the variable as bytea. Assuming you're using the ruby-pg driver the test_binary_values example from the driver should help you.
For bigger images (more than a few megabytes) use lo instead:
For bigger images please don't use bytea. It's theoretical max may be 2GB, but in practice you need 3x the RAM (or more) as the image size would suggest so you should avoid using bytea for large images or other large binary data.
PostgreSQL has a dedicated lo (large object) type for that. On 9.1 just:
CREATE EXTENSION lo;
CREATE TABLE some_images(id serial primary key, lo image_data not null);
... then use lo_import to read the data from a temporary file that's on disk, so you don't have to fit the whole thing in RAM at once.
The driver ruby-pg provides wrapper calls for lo_create, lo_open, etc, and provides a lo_import for local file access too. See this useful example.
Please use large objects rather than bytea.
* Incremental backup is possible with streaming replication, PITR / WAL archiving, etc, but again increasing the DB size can complicate things like WAL management. Anyway, unless you're an expert (or "brave") you should be taking pg_dump backups rather than relying on repliation and PITR alone. Putting images in your DB will also - by increasing the size of your DB - greatly slow down pg_basebackup, which can be important in failover scenarios.
† The adminpack offers local file access via a Pg connection for superusers. Your webapp user should never have superuser rights or even ownership of the tables it works with, though. Do your file reads and writes via a separate secure channel like WebDAV.
I'm writing an online tax return filing application using MVC3 and EF 4.1. Part of the application requires that the taxpayer be able to upload documents associated with their return. The users will be able to come back days or weeks later and possibly upload additional documents. Prior to finally submitting their return the user is able to view a list of files that have been uploaded. I've written the application to save the uploaded files to a directory defined in the web.config. When I display the review page to the user I loop through the files in the directory and display it as a list.
I'm now thinking that I should be saving the files to the actual SQL Server as binary data in addition to saving them to the directory. I'm trying to avoid what if scenarios.
What if
A staff member accidentally deletes a file from the directory.
The file server crashes (Other agencies use the same SAN as us)
A staff member saves other files to the same directory. The taxpayer should not see those
Any other scenario that causes us to have to request another copy of a file from a taxpayer (Failure is not an option)
I'm concerned that saving to the SQL Server database will have dire consequences that I am not aware of since I've not done this before in a production environment.
There's a really good paper by Microsoft Research called To Blob or Not To Blob.
Their conclusion after a large number of performance tests and analysis is this:
if your pictures or document are typically below 256K in size, storing them in a database VARBINARY column is more efficient
if your pictures or document are typically over 1 MB in size, storing them in the filesystem is more efficient (and with SQL Server 2008's FILESTREAM attribute, they're still under transactional control and part of the database)
in between those two, it's a bit of a toss-up depending on your use
If you decide to put your pictures into a SQL Server table, I would strongly recommend using a separate table for storing those pictures - do not store the employee foto in the employee table - keep them in a separate table. That way, the Employee table can stay lean and mean and very efficient, assuming you don't always need to select the employee foto, too, as part of your queries.
For filegroups, check out Files and Filegroup Architecture for an intro. Basically, you would either create your database with a separate filegroup for large data structures right from the beginning, or add an additional filegroup later. Let's call it "LARGE_DATA".
Now, whenever you have a new table to create which needs to store VARCHAR(MAX) or VARBINARY(MAX) columns, you can specify this file group for the large data:
CREATE TABLE dbo.YourTable
(....... define the fields here ......)
ON Data -- the basic "Data" filegroup for the regular data
TEXTIMAGE_ON LARGE_DATA -- the filegroup for large chunks of data
Check out the MSDN intro on filegroups, and play around with it!
I made a people maintenance screen. The client want me to store the photo of every person in the database, and I made it without problems. I have a separate table for the images with two fields, Id_person,and Image.
I'm a little worried because it's the first time that i work with images in database. will I have problems of performance when the table grows beyond 1000/5000 images? I suppose that the size of every image will make a difference. I'm sure that I will need to control that the user don't save very big images in the Database.
What would be a good size limit? The client only need pics of the face, but I'm sure that someone will try to make the pics with a "last model" camera in full quality ;)
Thanks.
It's usually preferred to keep a folder of images and the DB just references that folder. Ideally, each person has a unique ID and the files in the "images" folder match that ID.
If you really want to store the binary data directly, you can get a reasonable quality photo in 8KB of a JPEG (approx 250x250 pix # 25% quality). Of course, this would be unacceptable for printing, but is fine for identification.
Only you will know if you can accept an additional 8KB per row in your database server.
If you absoultely MUST do it this way, I would say limit it to just a few kilobytes each. However, every database admin in the world will probably tell you that blobing images into a database field is a very, very bad idea. Most noticably you will see the performance decrease drastically when the database file grows beyond two gigabytes in size.
I would prefer to do as jheddings said and have a folder with each person's ID be the file name and just use a standard .jpg or something after that on a network share so all computers using the app can access the images.
Some find that simply using the ID isn't good enough incase the photo needs to be deleted or archived, in which case they will put a NVARCHAR(MAX) field into their database and store the network file path to the image instead of the actual image.
I would only blob the image if your customer absolutely cannot have a network share path.
as long as it is in separate table with ID|BLOB only there shouldn't any performance issues fetching that photo, but on the other side i prefer keeping in DB only references to files on hdd (or even better if its only user photo you dont realy need a reference because user with ID 1 goes to /images/1.jpg)