Is This A Good Way To Do It?

[MrNewbie]

In my database program I'm planning to do things like this:

When the user enters a record that is written into a structure to be used in a linked list, and then its sent through a sorting algorithm to order the list elements.
When it is in order the ordered elements are written to the file sort of like this:

A 123 ER
B 312 DF
C 122 GF

If a new element is added later the file is read and the records in it are written into a linked list structure again, then the new record entry is sent through the sorting algorithm again to reorder the list with the new record.
When that is done, the disk file is totally cleared and the structures are written back to the disk file. And the process is repeated every time a new record is entered.
The same thing goes for deleting a record. The linked list is modified to not include the element and the file is cleared then the list is written back to the file.
For searching, the file is read again into linked list elements then compared with the search condition although this time the file is not cleared or written to at all.
So, is this a good way to do things?
I have the code to do pretty much everything, including rereading the tab-seperated file records back (Thanks to TheLinuxDuck). But to clear a file is the best way to delete and recreate it?
I'd like to hear any comments/suggestions.
Thanks
MrNewbie

[ 21 February 2001: Message edited by: MrNewbie ]

[error27]

that's a lot of reading and writing to disk.

but if it works do it. and then change it later if you want...

[Stuka]

The only suggestions I can think of are related to disk I/O and your sorting mechanism. For disk I/O, I'd only write out to disk when the user exits the program (or specifically chooses to write) rather than after every entry. This will save a TON of disk access. As for the sorting/storing, I'd use a list of linked lists - maybe based on some key (first letter/number, etc.) as the outer list, then link items from there.

[MrNewbie]

Are index files really necessary if speed is not important? I'd rather avoid using them if possible.

[TheLinuxDuck]

Originally posted by MrNewbie:
Are index files really necessary if speed is not important? I'd rather avoid using them if possible.

What I would suggest to you is to write a perl script to write out a large database file, something sized realisticly to about how big it could get..

Then, load the database with your script, and add an entry.

See how long it takes to sort and to rewrite the file. I think you'll be better off with an index if the database is going to be large. Having a large database loaded in memory can chew your memory away like a rabbit on a carrot.

Personally, I would suggest using an index file. As the database grows, you'll find that your write times for the database will be dramatically decreased. In fact, its worth the time savings.

As I said, though, if it's going to remain a small database, you'll prolly not have any troubles. :)

Try the create-a-database thing and add an entry to it. See how the script handles a large database file.
Maybe something like (based on your example):


#!/usr/bin/perl -w
use strict;
use warnings;

for(0..10000) {
print chr(65+($_%26)),"\t",
chr(48+($_%10)),
chr(48+(($_*2)%10)),
chr(48+(($_*3)%10)),"\t",
chr(65+(($_*2)%26)),
chr(65+(($_*3)%26)),"\n";
}
exit;


When I ran that for 10,000 records, the database was 90,009 bytes.

Anyhow.. :)

[MrNewbie]

Well its only gonna be 50 records at the most so would it be worth doing for just that small amount?

[Stuka]

Well, for only 50 records, I don't see that a flat file would be so bad - but I'd still cache my disk I/O operations - just as a matter of pride. :)

[TheLinuxDuck]

50 records? I wouldn't even give an index a second thought. Perl will whip right through that, so I won't worry about it.

One thing, though.. if there is a slight possibility that someone else may have the database file open when you're going to write out, then I would suggest locking the file so no one else can mess with it until you're done writing. Once you close the file handle, it will unlock.

Just my 3.141592653589793237462643383279 cents worth.

[MrNewbie]

Lock the file. Ok thanks I'll check that out.
MrNewbie