FileIO Library
At the April 2006 BRGroup Conference, Gabriel spoke to many of us about a new revolution in BR FileIO management. This topic generated so much interest that he was asked to present it again at the Fall Conference. There were also many suggestions for improvements to the library.
The FileIO Library began as a project to find a way to reduce the headache associated with making changes to your data files. In my experiences working for BRC, I had to make a change to the common file, a data file that stores information about communication with EDI VANs. I added some new fields, and removed a couple of old fields, and then I began searching through the BRC program suite to find and modify each program that referenced this data file in order to update it with the proper new form statement. This task quickly became daunting as I noticed that out of BRC’s program suite of over 600 programs, more than one third of them referenced the commun file. With 223 programs to modify, the task was given up as hopeless and tabled indefinitely.
I am happy to announce that we have solved this problem. When I have to make a change to a data file layout for my customers today, I can complete the task in under 1 minute, and not a single program needs to be changed in order to continue working with the new file layout. In fact, I don’t even have to update my customer’s data files, as the FileIO library takes care of this for me as well.
The trick is to define your file layouts in an ASCII text file layout file that I will tell you about in a minute. The FileIO Library will actually parse through your file layouts, and it will instruct your programs how to read the data file, so that you don’t have to. It will also automatically detect when you make changes to the file layout, and it will update your customer’s data files on the fly to make sure they have the latest version. Finally, it even contains a DataCrawler that you can use to examine the contents of any of your BR data files in a raw format.
FileIO 2 Improves upon FileIO in many ways. For information on FileIO 2, see the FileIO 2 page.
For more information about the FileIO Library visit the Sage AX Website
To download the latest copy of FileIO, click here.
The Method
I have created a file IO library that parses a formatted text file layout and uses this information to structure the opening and initializing of the reading of a file “object”. The word object here is used to refer to all the data in a given record in one of your files. This library is easy to use, and provided you follow some simple standards, it will simplify your lives immensely.
File Object Arrays
First, in your program you must create two matrices to store the file information. If we are dealing with the Color File these would be MAT COLOR$ and MAT COLOR. One will store all the string information about a color, and the other will store all the numeric data. Our working example will involve two files: a Color File and a Color Category File. You should dimension these to:
01030 DIM color$(1)*255,color(1) 01040 DIM colorcat$(1)*255,colorcat(1)
Forms Array
You will also need a variable to store the form statement associated with reading the files. This form statement will be dynamically generated whenever a new file is opened. It will say:
01020 DIM form$(1)*255
Library Linkage
You will also need the library call:
02020 library "fileio.br": fnopenfile, fnreaddescription$
fnOpen Function
Now, a simple function placed into all your files will simplify one aspect of this. Because BR libraries do not share variables with the programs they are called from, we will need to execute a simple proc file whenever we open a file to set the names of all our variables after we return. Create an FnOpen that calls the library version and runs the proc file when we are done. Notice the $$$ at the end of the procfile name. This will tell the procfile to self-destruct after execution. Since this procfile is used simply to return variable information back from the library to the main program, we don’t need it sitting around collecting dust.
99010 OPEN: ! ***** Function To Call Library Openfile And Generate Subs 99020 def Fnopen(Filename$*255, Mat F$, Mat F, Mat Form$; Inputonly, Keynum, Dont_Sort_Subs, Path$*255, Mat Descr$, Mat Field_Widths) 99030 dim _FileIOSubs$(1)*800 99040 let Fnopen=Fnopenfile(Filename$, Mat F$, Mat F, Mat Form$, Inputonly, Keynum, Dont_Sort_Subs, Path$, Mat Descr$, Mat Field_Widths, Mat _FileIOSubs$) 99050 for Index=1 to udim(mat _FileIOSubs$) : execute (_FileIOSubs$(Index)) : next Index 99060 fnend
Or, for those with Lexi:
OPEN: ! ***** Function To Call Library Openfile And Generate Subs
def Fnopen(Filename$*255, Mat F$, Mat F, Mat Form$; Inputonly, Keynum, Dont_Sort_Subs, Path$*255, Mat Descr$, Mat Field_Widths)
dim _FileIOSubs$(1)*800
let Fnopen=Fnopenfile(Filename$, Mat F$, Mat F, Mat Form$, Inputonly, Keynum, Dont_Sort_Subs, Path$, Mat Descr$, Mat Field_Widths, Mat _FileIOSubs$)
for Index=1 to udim(mat _FileIOSubs$) : execute (_FileIOSubs$(Index)) : next Index
fnend
Using FileIO in Your Programs
Now you are ready to process files. You simply open the file by saying:
04020 let colorfile=fnopen("color",mat color$,mat color,mat form$)
04030 let colorcatfile=fnopen("colorcat",mat colorcat$,mat colorcat,mat form$,1)
These statements tell the File Library to look in the filelay folder to find the file layout for the Color File, and read it to find out what the Color File looks like. Then Open the Color File, and set MAT COLOR$ and MAT COLOR to the correct size to hold an entire color record. Finally, it will define several variables that we can use as pointers in these arrays to read the data we want to see, and it will place the file handle into a variable called colorfile.
The second line above will do the same thing to the routing file, except the 1 parameter tells the function to open readonly. Because of the extra function we inserted into our program earlier, the subscripts array will be executed, creating the subscripts in memory, and the pointers (subscripts) will be set in the returned program. So, if I had a file layout such as the following:
color.dat, CO_, 1 color.key, CODE recl=127 =================================================== CODE$, Color Code, C 6 NAME$, English Name for Color, V 30 CATEGORY$, General Category of Color, C 6 HTML$, HTML Code for the Color, C 6
Then the functions would do the same thing as the following individual lines of code (assuming the next available file handle was 5):
DIM FORM$(5)*255 DIM COLOR$(9)*255, COLOR(1) OPEN #5: “Name=color.dat, kfname=color.key”,internal,outin,keyed LET FORM$(5)=”form C 6,V 30,V 6,C 6” LET FORM$(5)=CFORM$(FORM$(5)) LET COLORFILE=5 CO_CODE=1 CO_NAME=2 CO_CATEGORY=3 CO_HTML=4
Now we can read the file:
30120 read #colorfile, using form$(colorfile) : mat color$, mat color eof Ignore
and use the data:
30180 LET ColorCode$=color$(co_Code) !:
LET ColorName$=color$(co_Name)
The data is used by referencing the file array, with the subscript name, so the color’s description becomes color$(co_Name).
In the old days, if we wanted to change the file layout of our file, all of the programs that used that file would have to be changed one at a time to use the new file layout. Also, if the order of the fields changed, then all the programs would have to also be changed to use the new fields.
But now, using this function, we can change the file layout all we want. If we later want to insert a field into the file layout before color name, I won’t have to look at this program again; this program will just work fine, because the library tells it which fields to use.
Also, the code is easier to read and maintain.
Example
The whole thing looks like this:
01025 ! Dimension the Arrays
01030 DIM color$(1)*255,color(1)
01040 DIM colorcat$(1)*255,colorcat(1)
01050 DIM form$(1)*255
02000 !
02020 library "fileio.br": fnopenfile, fnreaddescription$
04020 let colorfile=fnopen("color",mat color$,mat color,mat form$) ! Open the file
05000 !
30120 read #colorfile, using form$(colorfile) : mat color$, mat color eof Ignore ! Read the file
30180 LET ColorCode$=color$(co_Code) !:
LET ColorName$=color$(co_Name) ! Use the data by referincing it in the file arrays
80000 !
90000 ! Every program using fileio needs the following code
99010 OPEN: ! ***** Function To Call Library Openfile And Generate Subs
99020 def Fnopen(Filename$*255, Mat F$, Mat F, Mat Form$; Inputonly, Keynum, Dont_Sort_Subs, Path$*255, Mat Descr$, Mat Field_Widths)
99030 dim _FileIOSubs$(1)*800
99040 let Fnopen=Fnopenfile(Filename$, Mat F$, Mat F, Mat Form$, Inputonly, Keynum, Dont_Sort_Subs, Path$, Mat Descr$, Mat Field_Widths, Mat _FileIOSubs$)
99050 for Index=1 to udim(mat _FileIOSubs$) : execute (_FileIOSubs$(Index)) : next Index
99060 fnend
File Layouts
Now let me demonstrate the anatomy of a properly formatted file layout. These should be placed in a subdirectory called filelay.
price.dat, PR_, 1 price.key, FARM price.ky2, ITEM price.ky3, FARM/ITEM/GRADE recl=127 =================================================== FARM$, Farm Code (or blank), C 4 ITEM$, Item Code, C 4 GRADE$, Quality, C 4 X, Empty, X 37 PRICE, Default Price, BH 3.2 COST, Default Cost, BH 3.2 XOPRICE, Default Christmas Price, BH 3.2 XOCOST, Default Christmas Cost, BH 3.2 MOPRICE, Default Mothers D Price, BH 3.2 MOCOST, Default Mothers D Cost, BH 3.2 VOPRICE, Default Valentine Price, BH 3.2 VOCOST, Default Valentine Cost, BH 3.2 DESCRIPTION$, Description of Price Rule, C 30
The first line contains the name of the Farm File, a unique prefix to add to all your subscript index pointers, and the file version number. The subscript value is to ensure that the program knows the difference between the Farm File’s Farm Code, and the Route File’s Route Code. (One will be RT_CODE and the other is FM_CODE). These are separated by a comma. Spacing does not matter, so adjust your spacing so that it looks nice.
The Version number is used to determine when the data has changed, and an update needs to be made to your data file. Your file layouts should all start at version 0, and each time you want to make a change, you may increment this value by 1.
If you look in the filelay folder, you will notice several files ending with a number. For example, you may see a color, a color.0, and a color.1 file. If you run the function and it can not find your data file (usually because this is a new file and it hasn’t been created yet), the function library will automatically create your file, based on the information you give in the first part of the file layout. Also, whenever you make changes to your file layout, the function library will automatically update the data files on disk for you. It does this by renaming the old file, creating a new one with the new version number, and then copying the data from the old one to the new one a record at a time.
Any time it creates a file, or updates the file on disk, it creates a backup of the file layout. The first time you run a program that tries to read your new data file, it will create the data file for you and make a backup, and if the number in your file layout is 0, then it will create, for example, a filelay\price.0 file, a backup of your file layout that the library uses to figure out what has changed the next time you try to update the file.
If you wish to make any changes to this data file, first you increment the version number, (in this case make the 0 into a 1). Then change the file layout all you want. You may rearrange fields, add new fields, add or remove keys, or change the record length.
Each time you try to open a file with the library, it reads the file version number of the file on disk (using the BR version() function), and then it compares it to the version number in your file layout. If your file layout has a higher version number then your existing data file, then the library opens the backup of the file layout for the version of the data file that exists on disk. It makes a backup copy of the existing data file, and creates a new file with the proper version number. Then it reads your records one at a time, and copies all the data from the old file into the new file one field at a time. If a field is dropped from the file layout, that data gets lost. If fields are rearranged, the data is copied and saved in the new file in the new positions. If a new field is added, it starts out blank (or 0).
The only step necessary for having your data files updated is to increment the version number every time you make a change to the file layout. You may make any changes you like to the file layouts, but do not change the names of your existing subscripts. If you change the subscript name, not only will all the programs that reference that subscript be broken, but additionally, the routine will not understand that you are changing the name. It will think you are dropping the old field and adding a new field and any data stored in this location will be lost when the file is updated.
price.dat, PR_, 1
The second line contains the name of the first key, and a definition telling what the key is based on. These are separated by a comma. The key definition is made up of each of the subscript names in this file that form the keyfields for the file. When the library is called to open a file, if the file does not exist, or if it needs to be updated, a new one will be created. The function will read these subscript names that form your key definitions, and it will calculate the proper kps and kln values. Then the create routine will use this information to create the new keys.
You notice the file can have as many keys as you want. The function will keep parsing key file names until it reaches the RECL= parameter. Remember it opens any OutIn files with all keys so that any changes you may make to the data stored on disk will be properly reflected in all the key files.
Also, notice the third key is based on three different fields. These are separated by a “/”. It is important that you use a “/” to separate your keys when you are building a key out of more then one field, because the function uses this “/” to help it make the proper BR syntax for defining complex keyfields.
The RECL= Parameter is also read and used whenever a new file is created or an old one is updated. It is also used to tell the function when to stop looking for key file names, and start looking for the rest of the data.
price.key, ITEM price.ky2, FARM price.ky3, ITEM/FARM/GRADE recl=127
The next line in the file is skipped by the parsing routine, and its purpose is to make the file layouts more readable to a programmer.
===================================================
Following file layouts, we will discuss data elements. There are three parameters on each line, and you make one line for each field in your file. The first parameter is the subscript index prefix code that you will use in your program to refer to this data element. Place a $ at the end of the subscript name for all string elements. Don’t place anything at the end of the subscript name for numeric elements.
The spaces are ignored, so you may include as many spaces as you wish to make the layout look good. However the file layout is designed to handle a maximum of 255 characters on each line, and a maximum of 80 characters for the Description names. Do not use tabs, however. For some reason, tabs confuse BR. The second parameter is the description. This description is for the benefit of the programmer, so when the programmer is reading the file layouts, (s)he can tell which field does what.
The description is also used by the built in DataCrawler, a program that reads any of your data files and displays the entire contents in raw form in a listview. The Descriptions become the headings for each column in the listview. Imagine this as something that you would show to a programmer, or a highly technical user who needed to investigate a data error of some kind.
The third parameter is the form statement, which is pretty straightforward. Any items with a form statement of type “X” will be ignored, except that the length will still be used to calculate the position on disk of all remaining fields in the record. The library will take X fields into consideration when building the form statement, but not at any other time.
FARM$, Farm Code (or blank), C 4 ITEM$, Item Code, C 4 GRADE$, Quality, C 4 DUMMY, Empty, X 37 PRICE, Default Price, BH 3.2 COST, Default Cost, BH 3.2 XOPRICE, Default Christmas Price, BH 3.2 XOCOST, Default Christmas Cost, BH 3.2 MOPRICE, Default Mothers D Price, BH 3.2 MOCOST, Default Mothers D Cost, BH 3.2 VOPRICE, Default Valentine Price, BH 3.2 VOCOST, Default Valentine Cost, BH 3.2 DESCRIPTION$, Description of Price Rule, C 30
And that’s all there is to it.
The Library (Function Reference)
The library includes a number of useful functions. I hope you find these to be as useful for you as I have found them to be for me.
fnSettings (Global Settings Code)
The newest version of the FileIO library supports some features that require you to specify Global Settings values. These are done by modifying the contents of the fnSettings routine at the beginning of the fileIO library.
01010 SETTINGS: ! ***** Set Global Defaults For The Library Here 01020 def Fnsettings(&Defaultfilelayoutpath$;&Promptonfilecreate,&Createlogfile,&EnforceDupkeys,&StartFileNumber) 01030 let EnforceDupkeys=1 ! Enforce that key number 1 is unique key 01040 let Defaultfilelayoutpath$="filelay\" ! Path To Your File Layouts 01050 let Promptonfilecreate=1 ! Ask User Before Creating New Files 01060 let Createlogfile=0 ! Create the logfile 01070 let StartFileNumber=1 ! File IO will use file numbers above this number 01080 let CheckIndex=0 ! Automatically Update Indexes 01080 fnend
EnforceDupkeys
Turn on the EnforceDupkeys option to force that the first key listed in your file layouts is a unique key. FileIO will generate the standard BR error for Dupkeys when attempting to create indexes if it is not unique.
DefaultFileLayoutPath
Use the DefaultFileLayoutPath option to specify the path from your programs to your file layout folder. The default is "filelay\". Use this setting if you want to place your file layouts in another folder from the default.
PromptOnFileCreate
Use the PromptOnFileCreate setting to cause FileIO to put up a message box whenever its attempting to create a new file. This happens during the automatic update procedure, as well as during any attempt to access a file that does not exist. This setting should be turned off in your live system, and only turned on for development purposes. If you use the message box to cancel creating of the new file, then the file is not created, and fileIO or your application will most likely give an error when you attempt to actually access the new file.
It can be useful during development to make sure that you don't accidentally create the wrong files, due to an incorrectly specified path or a typeo in the filename in the layout file. However, in a live system, these things have already been tested, and you generally want the default behavior, because you don't want your users to have the ability to cancel the normal operation of FileIO.
CreateLogFile
Use this option to specify weather or not to use the FileIO log file. If it is turned on, a log file called FileIO.log in the current directory is created with entries listing all attempts to open a file, automatically update one, or automatically update your indexes.
StartFileNumber
Use this option to set the starting file number that the fnGetFileNumber and the fnOpen functions use to search for available file numbers. This is so that if you have certian reserved file numbers in your code, you can make sure that FileIO will not use those reserved file numbers, causing a conflict with your already existing programs. The default is 1, which is fine if your software does not have any reserved file numbers.
CheckIndex
This function is used for Partial FileIO Implementations. If all your software uses FileIO then all of your indexes are kept automatically up to date by the FileIO library and BR's internal file processing systems. However, if some of programs do not use FileIO, and you use the FileIO library to add a new index file that those other programs do not know about, then its possible for the additional index file to become out of date when the master file is updated by your other programs.
Use this setting to cause FileIO to check the DateTime stamp on all your index files when opening a new file, and automatically update any indexes that may have potentially gotten out of date from other programs.
This option slows down the processing of the fnOpen function, particularly when running under client server over the internet. If you know that every program in your application suite uses FileIO, and that any programs that do not use FileIO still properly update all the indexes that your data file uses, then you can safely leave this setting turned off, and take advantage of a significant speed increase when opening a lot of files using the fileIO system.
fnOpenFile
The primary function opens a file, and it’s used like so:
fnOpenFile(Layout$,MAT F$,MAT F,MAT Form$;InputOnly,KeyNum,DontSortSubs,Path$,mat Description$,mat FieldWidths,SupressPrompt)
- FileName$ - The filename of the file layout for the file you’re reading.
- MAT F$ - The array that will be used to hold the string data for the file.
- MAT F – The array that will hold the numeric data for the file.
- MAT Form$ - An array of form statements.
- InputOnly – 1 means open for input only. 0 means open for OutIn and open every key file associated with the given data file (this is because all key files need to be open for the keys to be updated on an output) (defaults to 0).
- KeyNum – This tells the function of which key to return the file handle (defaults to 1).
- DontSortSubs – The function by default will sort the string and numeric subs in order to allow for reading the data into an array, and this parameter would turn this functionality off. However, if you are trying to read your data without reading it into an array, you are missing the point. You should leave this option turned off (0). This is a totally unsupported feature, and should always be set to 0, if it is specified at all.
- Path$ - The path to your data files. This optional parameter can be passed in by the calling function. It is appended to the beginning of the paths given in the file layout files. It is useful when your data files can be in different locations depending on the state of your program.
- mat Description$ - This optional parameter provides a way to read the description for each field from the original file layout. If the parameter is not provided, no description data will be returned.
- mat Fieldwidths – This optional parameter will return an array of the calculated display field widths. This number will always be at least long enough to contain the data for this field.
- SupressPrompt - This optional parameter can be used to supress the prompt normally given when fileIO creates a file. It can have three values. A value of 0, the default, uses the setting stored in your FileIO fnSettings routine to determine weather or not to prompt on Creation of a new file. A nonzero value always supresses the prompt. A value of 1 indicates never to create a file if the file doesn't exist. A value of 2 automatically creates the file if the file doesn't exist.
When using fnOpenFile, you really want to call your local function FnOpen, which in turn calls fnOpenFile for you.
Example:
Let FFILE=FNOPEN(“FARM”,MAT FARM$,MAT FARM,MAT FORM$,0,2) Let RFILE=FNOPEN(“ROUTE”,MAT ROUTE$,MAT ROUTE,MAT FORM$,1,3) Let CFILE=FNOPEN(“CUSTOMER”,MAT CUSTOMER$,MAT CUSTOMER,MAT FORM$)
assuming:
farm.dat has 3 keys: farm.ky1, farm.ky2, farm.ky3 route.dat has 4 keys: route.ky1 … route.ky4 customer.dat has 2 keys: customer.ky1, customer.ky2
This will perform the following opens and set the following variables:
OPEN #1: “Name=Farm.Dat, kfname=farm.ky2”,internal,outin,keyed OPEN #2: “Name=Farm.Dat, kfname=farm.ky1”,internal,outin,keyed OPEN #3: “Name=Farm.Dat, kfname=farm.ky3”,internal,outin,keyed OPEN #4: “Name=Route.Dat, kfname=route.ky3”,internal,input,keyed OPEN #5: “Name=Customer.Dat,kfname=customer.ky1”,internal,outin,keyed OPEN #6: “Name=Customer.Dat,kfname=customer.ky2”,internal,outin,keyed LET FFILE=1 LET RFILE=4 LET CFILE=5
This will also resize the arrays, set the form statement, and create all the subscripts to make reading and writing clear and simple, as in the above example. The KEYNUM parameter is where you list the key file you would like to use for input and output. The extra keys are opened as a courtesy to you to ensure that all keys get updated properly when the file is changed.
Example:
DIM FORM$(1),PRICE$(1)*255,PRICE(1), DIM DESCRIPTION$(1)*255,FIELDWIDTHS(1)
Let PRICEFILE=FNOPENFILE(“PRICE”,MAT PRICE$,MAT PRICE,MAT FORM$,0,2,0,MAT DESCRIPTION$)
Assuming the Price File layout (filelay\price) contains:
price.dat, PR_, 1 price.key, FARM price.ky2, ITEM price.ky3, FARM/ITEM/GRADE recl=127 =================================================== FARM$, Farm Code (or blank), C 4 ITEM$, Item Code, C 4 GRADE$, Quality, C 4 DUMMY, Empty, X 37 PRICE, Default Price, BH 3.2 COST, Default Cost, BH 3.2 XOPRICE, Default Christmas Price, BH 3.2 XOCOST, Default Christmas Cost, BH 3.2 MOPRICE, Default Mothers D Price, BH 3.2 MOCOST, Default Mothers D Cost, BH 3.2 VOPRICE, Default Valentine Price, BH 3.2 VOCOST, Default Valentine Cost, BH 3.2 DESCRIPTION$, Description of Price Rule, C 30
This will perform the following opens and set the following variables:
OPEN #1: “Name=Price.Dat, kfname=Price.ky2”,internal,outin,keyed OPEN #2: “Name=Price.Dat, kfname=Price.ky1”,internal,outin,keyed OPEN #3: “Name=Price.Dat, kfname=Price.ky3”,internal,outin,keyed let PRICEFILE=1 let PR_FARM=1 let PR_ITEM=2 let PR_GRADE=3 let PR_DESCR=4 let PR_PRICE=1 let PR_COST=2 let PR_XOPRICE=3 let PR_XOCOST=4 let PR_MOPRICE=5 let PR_MOCOST=6 let PR_VOPRICE=7 let PR_VOCOST=8 MAT FORM$(1) MAT PRICE$(4) MAT PRICE(8) MAT DESCRIPTION$(12) MAT FIELDWIDTHS(12) let FORM$(1)=CFORM$(”form C 4,C 4,C 4,POS 74,C 30,POS 50,BH 3.2,BH 3.2,BH 3.2,BH 3.2,BH 3.2,BH 3.2,BH 3.2,BH 3.2”) let Description$(1)= “Farm Code (or blank)” let Description$(2) = “Item Code” let Description$(3) = “Quality” let Description$(4) = “Description of Price Rule” let Description$(5) = “Default Price” let Description$(6) = “Default Cost” let Description$(7) = “Default Christmas Price” let Description$(8) = “Default Christmas Cost” let Description$(9) = “Default Mothers D Price” let Description$(10) = “Default Mothers D Cost” let Description$(11) = “Default Valentine Price” let Description$(12) = “Default Valentine Cost” let FieldWidths(1) = 4 let FieldWidths(2) = 4 let FieldWidths(3) = 4 let FieldWidths(4) = 30 let FieldWidths(5) = 8 let FieldWidths(6) = 8 let FieldWidths(7) = 8 let FieldWidths(8) = 8 let FieldWidths(9) = 8 let FieldWidths(10) = 8 let FieldWidths(11) = 8 let FieldWidths(12) = 8
There are a few things I’d like to point out about the example above. First, you will notice that the form statement jumps around to put all the strings first, and then the numbers last. This is why it has a “POS 74, C 30,POS 50”. Then notice that the subscripts for the string variables are 1 .. 4, and the subscripts for the numbers are 1 .. 8. Finally, the order of the Description and FieldWidth fields also match the sorted positioning of the Form Statement.
The reason that we sort our form statements is so that BR will allow us to read the file into arrays by saying:
Read #pricefile, using form$(pricefile) : mat price$, mat price
Without resorting, the above statement would give a conversion error as BR attempted to put the PR_PRICE field inside mat price$(4). However, because we have reordered the form statement, we are able to read them safely into two arrays, and then we will be able to use the values without caring what position they are in the file, by simply saying PRICE$(PR_DESCRIPTION) when we want the description, and PRICE(PR_PRICE) when we want the pr_Price field.
Another thing you may notice about the above example is that it gives the calculated display length for the numeric fields as 8, when on the disk (and in the file layout) they are listed as BH 3.2. The reason for this is the program looks at the BH 3, and figures that a number that takes up 3 bytes on disk has a potential maximum size of 256**3 or 16,777,216, and therefore will need up to 8 characters of screen display space to view.
Finally, note that any field with a form statement of X is ignored by the library, except that the blank space is used when building the FORM statement.
FnOpenFile and FnOpen have been improved to pass the subscripts back in an array instead of a proc file, which is much faster, and also not as prone to causing file sharing errors.
fnCloseFile
This function will close the specified filenumber and all keys that were opened with the file.
You should use it only for files that were opened using the library for OutIn.
The purpose of this function is to facilitate the closing of the extra keys that are opened when you open a file for OutIn with the library.
You must give the function the name of the file layout. It uses this information to determine how many keys were opened, and how many it must therefore close.
Then it basically starts at the file number you gave it, and closes the next X files that were opened with the same file name as this, where X is the number of keys associated with this file.
The syntax is:
fnCloseFile(filenumber,filelay$;path$)
- FileNumber – The filenumber of the file you are trying to close.
- FileLay$ - The name of the file layout for this file. This is used to determine how many keys the file would have been opened with and therefore how many we now need to close.
- Path$ - Optional Alternate Path. Use to close files that were opened using the open file's optional alternate path parameter.
fnMakeSubProc
The standard FNOpenFile routine opens a file and sets several values, and also passes back the subscripts to the calling program. The fnMakeSubProc$ routine passes back the subs without actually opening the file. This is useful when a file record is passed as arrays into a library function in another program, or when you chained to another program and you need to know how to reach the data in the arrays without actually reopening the file.
fnMakeSubProc(filelay$;mat Subscripts$)
- FileLay$ - The name of the file layout from which to read the subscripts.
- mat Subscripts$ - If you give this optional array, fnMakeSubProc passes the subscripts back in this array rather then in the subs.$$$ file. This is much faster and helps avoid sharing conflicts.
When this routine returns, you can set the subscripts in your program by executing every element of the Subscripts$ array in a for/next loop.
If you did not pass in a subscripts array, then the old subs.$$$ file is created for backwards compatibility. If you proc that file, the proper subscripts will be set.
fnReadLayoutArrays
This function reads the fields in a file layout into arrays. You call it like the following
fnReadLayoutArrays(filelay$,&prefix$;mat SSubs$, mat NSubs$, mat SSpec$, mat NSpec$,mat SDescription$, mat NDescription$,Mat Spos,Mat Npos)
- filelay$ - the name of the layout to read
- prefix$ - the return value for the prefix for the file
- mat SSubs$ - the return value for all the string subscripts in the layout
- mat NSubs$ - the return value for the numeric subscripts in the layout
- mat SSpec$ - the return value for the string fields specs
- mat NSpec$ - the return value for the numeric fields specs
- mat SDescription$ - the return value for the Descriptions of the String Specs
- mat NDescription$ - the return value for the Descriptions of the Numeric Specs
- mat SPos - the return value for the starting positions of the String Specs
- mat NPos - the return value for the starting positions of the Numeric Specs
This function call would read the fields from the layout in filelay$, and it would return the prefix to prefix$. The Subs would be returned in mat SSubs$ and mat NSubs$.
The Spec statements are returned in mat SSpec$ and mat NSpec$ and the Descriptions are returned in mat SDescription$ and mat NDescription$. The start positions on disk of each element are returned in mat SPos and mat NPos.
All the arrays are optional. If you don’t pass them the information they supposed to record is simply not returned.
fnReadLayoutHeader
This function reads the header for a given file layout.
fnReadLayoutHeader(Layoutname$*255;&Filename$,Mat Keys$,Mat KeyDescription$)
- LayoutName$ - Name of the layout to read
- Filename$ - The file name read from the layout
- Mat Keys$ - Array to be populated with the list of keys for the file
- Mat KeyDescription$ - Key Description String returned from the file
fnReadEntireLayout
This function reads the entire layout by calling fnReadLayoutHeader and fnReadLayoutArrays.
fnReadEntireLayout(Layoutname$*255;&Filename$,&Prefix$,Mat Keys$,Mat KeyDescription$,Mat Ssubs$,Mat Nsubs$,Mat Sspec$,Mat Nspec$,Mat Sdescription$,Mat Ndescription$,Mat Spos,Mat Npos)
- LayoutName$ - Name of the layout to read
- Filename$ - The file name read from the layout
- prefix$ - the return value for the prefix for the file
- Mat Keys$ - Array to be populated with the list of keys for the file
- Mat KeyDescription$ - Key Description String returned from the file
- mat SSubs$ - the return value for all the string subscripts in the layout
- mat NSubs$ - the return value for the numeric subscripts in the layout
- mat SSpec$ - the return value for the string fields specs
- mat NSpec$ - the return value for the numeric fields specs
- mat SDescription$ - the return value for the Descriptions of the String Specs
- mat NDescription$ - the return value for the Descriptions of the Numeric Specs
- mat SPos - the return value for the starting positions of the String Specs
- mat NPos - the return value for the starting positions of the Numeric Specs
fnReadSubs
This function reads the subscripts from a layout into Subs arrays.
fnReadSubs(Layout$,mat SSubs$,mat NSubs$,&Prefix$)
- Layout$ - the file layout name
- mat SSubs$ - the String Subscript Names
- mat NSubs$ - the Number Subscript Names
- Prefix$ - the files Prefix
fnReadKeyFiles
This function reads the list of key files from the layout.
fnReadKeyFiles(Layout$,mat Keys$)
- Layout$ - The file layout
- mat Keys$ - output array, the keys are returned here.
fnReadDescription$
In the example program I used above, I am reading the Color File to get details about each color. The color file contains a colorcat code field, but I want to display the colorcat description. The colorcat file contains a few details about a color category, and it is indexed based on colorcat code:
route.dat, RT_ route.key, CODE recl=512 =================================================== CODE, Routing Code, C 6 NAME, Routing Name Description, C 30 MOFT, T/A/C (truck/air/courier), C 1 SHIPPINGDAY, Day of Week for Shipping, C 7
Now, as you know, in the above example, we have already opened the ColorCat File, and we have opened and read a Color record.
04020 let colorfile=fnopen("color",mat color$,mat color,mat form$)
04030 let colorcatfile=fnopen("colorcat",mat colorcat$,mat colorcat,mat form$,1)
30120 read #colorfile, using form$(colorfile) : mat color$, mat color eof Ignore
30180 LET ccode$=color$(CO_CODE) !:
LET Name$=color$(CO_NAME)
Now all that’s left to do is to take the Color File’s ColorCat code and use it to look up the ColorCat File’s description.
30190 LET ColorCat$ = fnReadDescription$(ColorCatFile, CC_NAME, Color$(CO_CATEGORY), mat ColorCat$, mat ColorCat, mat form$)
Lets take a look at how this function works: fnReadDescription$(Fl,Subscript,key$,mat F$,mat F,mat Form$)
- Fl – File Handle for file to use (Route File).
- Subscript – Index of field in record to use (should be taken from file layout) (RT_NAME, which should equal 2 after opening routefile (unless we change the file layout later)).
- key$ - Item that should match a key in this file somewhere (in this case it is the route code from the farm record).
- mat F$ & mat F - Work Variables to hold a file record from the file we are reading they have to be dimensioned properly, which is why we use our colorcat$ and our colorcat for these parameters.
- mat Form$ - This will need to be our array of form statements, so the function can read the file properly.
fnReadUnopenedDescription$
This function accomplishes the same thing as fnReadDescription except that it opens the data file for you. It is slower then FnReadDescription$, but it is easier to use.
FnReadUnopenedDescription$(Layout$,key$*255;Field)
- Layout$ - the layout of the file we are reading
- key$ - the key of the record to be read
- Field - the element of the data file to return. If not given, this defaults to 2, so sometimes it is a good idea to design your data files so that string field number two is a descriptive field, like Name or Description.
fnReadNumber
FnReadNumber does the same thing that ReadDescription does except it reads a numeric field instead of a description.
Lets take a look at how this function works:
FNREADNUMBER(Fl,subscript,key$,mat F$,mat F,mat fm$)
FL – File Handle for file to use (Route File). SUBSCRIPT – Index of field in record to use (should be taken from file layout) (RT_NAME, which should equal 2 after opening routefile (unless we change the file layout later)). KEY$ - Item that should match a key in this file somewhere (in this case it is the route code from the farm record). MAT F$ & MAT F - Work Variables to hold a file record from the file we are reading – they have to be dimensioned properly, which is why we use our colorcat$ and our colorcat for these parameters. MAT FM$ - This will need to be our array of form statements, so the function can read the file properly.
fnReadUnopenedNumber
This function accomplishes the same thing as fnReadNumber except that it opens the data file for you. It is slower then FnReadNumber, but it is easier to use.
FnReadUnopenedNumber(Layout$,key$*255;Field) Layout$ - the layout of the file we are reading key$ - the key of the record to be read Field - the element of the data file to return. If not given, this defaults to 2, so sometimes it is a good idea to design your data files so that string field number two is a descriptive field, like Name or Description.
fnReadRelativeDescription$
This function is the same as fnReadDescription$ but for Relative Files.
FnReadRelativeDescription$(FileNumber,SubscriptToRead,RecordNumber,mat F$,mat F,mat form$)
fnReadRelUnopenedDescription$
This is the same as ReadUnopenedDescription$ but for Relative Files.
fnReadRelUnopenedDescription(LayoutName$,RecordNumber;Field)
fnReadRelativeNumber
This is the same as fnReadNumber but for Relative Files.
fnReadRelativeNumber(FileNumber,SubscriptToRead,RecordNumber,mat F$,mat f,mat Form$)
fnReadRelUnopenedNumber
This is the same as fnReadUnopenedNumber but for Relative Files.
fnReadRelUnopenedNumber(LayoutName$,RecordNumber;Field)
fnGetFileNumber
FnGetFileNumber is a simple function to find a valid unused file handle. If you use this function in all of your programs, they will become much more portable, as you will never have to worry about your file handles fighting with each other. The function will return 0 if no free file number was found (but with 999 of them available now, I have never seen it happen).
FNGETFILENUMBER(;X,Count)
X – This optional parameter will specify where to start looking. Count - This optional parameter will specify how many file numbers to find in a row. If count is 3, then fnGetFileNumber will return the first filenumber in the first gap of three unused file numbers that it finds.
fnUniqueKey
This function tests to see if a given key is unique to the specified file. This function is very useful for situations where you need to ensure that the user-entered key is unique.
FnUniqueKey(Fl,key$*255)
FL – The file number of the opened file. Key$ - This is the key to test. If this key is the key for a preexisting record in the file, the function will return false. If this key can not be found in the file, the function will return true.
fnReadAllKeys
This function will read through a file, returning the specified element(s) from each record into (a) dynamically dimensioned array(s). For example, I could tell it to give me the Inventory Code for every inventory item in my database in one array, while placing the Inventory Description (name) for each item into the corresponding position in another array.
FnReadAllKeys(Fl,mat f$,mat f,mat fm$,sub1,mat out1$;sub2,mat out2$)
FL – The file handle for the already opened file in question. MAT F$ - Work array with a size that corresponds to the number of string elements in the record (this is calculated automatically in the Open statement, if you used this library to open the file). MAT F – Work array with a size that corresponds to the number of numeric elements in the record (this is calculated automatically in the Open statement, if you used this library to open the file). MAT fm$ - Array of Forms statement. FM$(FL) must be the form statement for this file (this is calculated automatically in the Open statement, if you used this library to open the file). Sub1 – Subscript of the element to return in the first array MAT out1$ - Array in which to return the data. This array will be resized to match the number of records in the file. Sub2 – Subscript of the element to return in the second array. This is optional. If it is specified, you must give MAT out2$, or BR will return an error. MAT out2$ - Array in which to return the second (optional) parameter. This array will be resized to match the number of records in the file. This parameter MUST be provided when Sub2 is given (non-zero). This parameter will not be used if Sub2 is not given or is given as 0.
If I wanted to implement the above example, I would say:
FnReadAllKeys(InventFile,mat Invent$,mat Invent,mat Form$,IN_Code,mat InvtList$,IN_Name,mat InvtNames$)
fnReadMatchingKeys
This function will read through a file, returning the specified element(s) from each record where the key matches the specified key into (a) dynamically dimensioned array(s). This function is the same as the above function except this one will filter the results, returning only those records where the key matches the specified key.
FnReadMatchingKeys(Fl,mat f$,mat f,mat fm$,key$,keysub,sub1,mat out1$;sub2,mat out2$)
FL – The file handle for the already opened file in question. MAT F$ - Work array with a size that corresponds to the number of string elements in the record (this is calculated automatically in the Open statement, if you used this library to open the file). MAT F – Work array with a size that corresponds to the number of numeric elements in the record (this is calculated automatically in the Open statement, if you used this library to open the file). MAT fm$ - Array of Forms statement. FM$(FL) must be the form statement for this file (this is calculated automatically in the Open statement, if you used this library to open the file). Key$ - Only records where the key field matches key$ will be returned. Keysub – This tells the function which element is the key element for this particular file number. Sub1 – Subscript of the element to return in the first array. MAT out1$ - Array in which to return the data. This array will be resized to match the number of records in the file. Sub2 – Subscript of the element to return in the second array. This is optional. If it is specified, you must give MAT out2$, or BR will return an error. MAT out2$ - Array in which to return the second (optional) parameter. This array will be resized to match the number of records in the file. This parameter MUST be provided when Sub2 is given (non-zero). This parameter will not be used if Sub2 is not given or is given as 0.
fnReadAllNewKeys
This function will read through a file, returning the specified element(s) from each record that isn’t already there into (a) dynamically dimensioned array(s). This function is the same as the above function, except this one will filter the results returning only those records that don’t already exist in the array (eliminating duplicates).
FnReadAllNewKeys(Fl,mat f$,mat f,mat fm$,sub1,mat out1$; dont_reset,sub2,mat out2$)
FL – The file handle for the already opened file in question. MAT F$ - Work array with a size that corresponds to the number of string elements in the record (this is calculated automatically in the Open statement, if you used this library to open the file). MAT F – Work array with a size that corresponds to the number of numeric elements in the record (this is calculated automatically in the Open statement, if you used this library to open the file). MAT fm$ - Array of Forms statement. FM$(FL) must be the form statement for this file (this is calculated automatically in the Open statement, if you used this library to open the file). Sub1 – Subscript of the element to return in the first array. MAT out1$ - Array in which to return the data. This array will be resized to match the number of records in the file. Dont_reset – By default the array will clear the contents of the arrays that are passed in. This Boolean flag will instruct the function to not empty the passed in arrays. Sub2 – Subscript of the element to return in the second array. This is optional. If it is specified, you must give MAT out2$, or BR will return an error. MAT out2$ - Array in which to return the second (optional) parameter. This array will be resized to match the number of records in the file. This parameter MUST be provided when Sub2 is given (non-zero). This parameter will not be used if Sub2 is not given or is given as 0.
FnReadFilterKeys
This function by Mikhail Zheleznov is a modification of the above functions. Like its predecessors, it reads an entire file, populating the specified arrays with data from all or some of the records in the file. The given subscripts specify which fields to return from each record. The key given specifies search criteria for the records. The filter specifies filtering criteria that can be preformed on the data before it is returned.
FnReadFilterKeys(Fl,Mat F$,Mat F,Mat Fm$,Key$,Keyfld,Sub1,Mat Out1$;Filter$,Filter_Sub, Readlarger,Sub2,Mat Out2$, Sub3, Mat Out3$, Sub4,Mat Out4$)
FL – The file handle for the already opened file in question. MAT F$ - Work array with a size that corresponds to the number of string elements in the record (this is calculated automatically in the Open statement, if you used this library to open the file). MAT F – Work array with a size that corresponds to the number of numeric elements in the record (this is calculated automatically in the Open statement, if you used this library to open the file). MAT fm$ - Array of Forms statement. FM$(FL) must be the form statement for this file (this is calculated automatically in the Open statement, if you used this library to open the file). Key$ - The key to search for in the read statements Keyfld – the subscript of the key field in the data file. This must match the key field specified in the data file otherwise the function won’t work. Sub1 – Subscript of the element to return in the first array. MAT out1$ - Array in which to return the data. This array will be resized to match the number of records in the file. Filter$ – This optional parameter identifies matches to search for in the records. It works in conjunction with Filter_Sub Filter_Sub – This parameter specifies which field to look for in the records for matches to Filter$. Only records which have Filter$ in the specified field will be returned. Sub2 – Subscript of the element to return in the second array. This is optional. If it is specified, you must give MAT out2$, or BR will return an error. MAT out2$ - Array in which to return the second (optional) field. This array will be resized to match the number of records in the file. This parameter MUST be provided when Sub2 is given (non-zero). This parameter will not be used if Sub2 is not given or is given as 0. Sub3 – Subscript of the element to return in the third array. This is optional. If it is specified, you must give MAT out3$, or BR will return an error. MAT out3$ - Array in which to return the third (optional) field. This array will be resized to match the number of records in the file. This parameter MUST be provided when Sub3 is given (non-zero). This parameter will not be used if Sub3 is not given or is given as 0. Sub4 – Subscript of the element to return in the fourth array. This is optional. If it is specified, you must give MAT out4$, or BR will return an error. MAT out4$ - Array in which to return the fourth (optional) field. This array will be resized to match the number of records in the file. This parameter MUST be provided when Sub4 is given (non-zero). This parameter will not be used if Sub4 is not given or is given as 0.
This function was written by Mikhail Zheleznov for the fileIO library.
fnMakeUniqueKey$
This function generates a unique key for a given file. This is useful if you do not care what the key is, but it needs to be unique. I use this function in situations where the user never needs to know what the given key is.
This function reads the key length for the given file. Then it returns a string that is the right length, which is generated by counting in binary until a key is found that does not appear in the file. The first key found for a 4 byte key field would be chr$(0)&chr$(0)&chr$(0)&chr$(0). If that key was already in use in the file, the function would return chr$(0)&chr$(0)&chr$(0)&chr$(1). If that one was also in use, it would then try chr$(0)&chr$(0)&chr$(0)&chr$(2), and so on until it found one that wasn’t in use.
FnMakeUniqueKey$(fl)
FL – This is the file number of the opened file for the desired key.
fnLog & fnErrLog
These next two functions are functions to aid in logging. When you call either of these two functions, you give them a string that you wish to log. They will open a textfile called FileIO.log and add a line to the end of it containing some user information such as login_name and session$, and the string that you specified. FnErrLog does the same thing as fnLog except it also logs the Error Number and the Line.
The syntax is:
FnLog(string$) FnErrLog(String$)
fnNotInFile
This function will determine if a non-key element in a line is unique. It works almost exactly like fnUniqueKey specified above, except that it allows you to enter the subscript value of the element you are checking for uniqueness. You can use this to look for uniqueness in any field, not just in the key field. Additionally, the search engine used by this function looks for a partial match, and will only return true if the given string is not found anywhere in the specified element for the given file.
FnNotInFile(string$*100,filename$,sub)
String$ - Value to check for uniqueness in this file. Filename$ - This is the name of the file layout of the file you want to check. This file does not have to be open in order for you to search it, because the function will open it for you. Sub – This is the subscript value of the element you are checking.
fnReadLayouts
This function reads the file layout folder and returns all the applicable layouts in the passed in array.
FnReadLayouts(mat Dirlist$)
Mat Dirlist$ - After running the function, mat Dirlist$ will contain a list of all the file layouts that FileIO can find.
fnDoesLayoutExist
This function returns true if the given file layout exists. It returns false if the layout does not exist.
FnDoesLayoutExist(layout$)
Layout$ - Layout to look for
fnKey$
This function formats the key for the given file number.
FnKey$(FileNumber, Key$) FileNumber - the file number we are formatting the key for Key$ - the key we are trying to format.
fnBuildKey$
This function will return they key for a given record in a data file. It actually reads the file layout and builds the key to match the layout.
FnBuildKey$(Layout$, Mat F$, Mat F; Keynum) Layout$ - the name of the data file to build the key for. Mat F$ - the string array of the file object Mat F - the numeric array of the file object KeyNum - This optional parameter specifies which of the key files in the given layout the key should be built for.
fnReadRecordWhere$
This function returns the record where the given element matches the given value. It does not depend on any key files, and it can be used to locate a record based on any element. However, it loops through the entire data file to do this and it could be a bit slow for large data files. This function opens the file for you, so the file does not have to be already opened.
FnReadRecordWhere$(Layout$,SearchSub,SearchKey$*255,ReturnSub) Layout$ - the layout of the file we are searching SearchSub - Subscript of the element to look in SearchKey$ - The value we are looking for ReturnSub - Subscript of the element to return
fnUpdateFile
This function checks and Updates a data file if it needs to be updated, by opening and then closing the data file.
fnUpdateFile(FileLayout$) FileLayout$ - The name of the file to update
fnDisplayLength
This function calculates the display length of a field based on its form spec.
fnDisplayLength(Spec$) Spec - the Form Spec to return the display length of.
fnLength
This function calculates the length on disk of a field based on its form spec.
fnLength(Spec$) Spec - the Form Spec to return the display length of.
fnDataCrawler
This function launches the Data Crawler as a function, so you can make your own programs link to it. Special thanks to Mikhail Zheleznov for turning the DataCrawler into a library function.
fnDataCrawler(Layout$;SRow$,SCol$,Rows$,Cols$)
fnDataEdit
This function launches the DataGrid as a function, so you can make your own programs link to it. Special thanks to Mikhail Zheleznov for turning the DataGrid into a library function.
fnDataEdit(Layout$;SRow$,SCol$,Rows$,Cols$)
fnShowData
This function builds a list or a grid in a floating window that is tied to a data file. It uses the same function that the datacrawler uses, but its much more customizable. All other datacrawler functions are just wrappers for this one.
The first parameter is the only required parameter.
def library fnShowData(FileLay$;Edit,sRow,sCol,Rows,Cols,KeyNumber,Caption$*127,Path$*255,KeyMatch$*255,SearchMatch$*255,CallingProgram$*255,mat Records,mat IncludeCols$,mat IncludeUI$,mat ColumnDescription$,mat ColumnWidths,mat ColumnForms$,DisplayField$*80,mat FilterFields$,mat FilterForm$,mat FilterCompare$,mat FilterCaption$,mat FilterDefaults$,mat FilterKey) FileLay$ - the file layout you want to display Edit - 1 for Edit (Grid) and 0 for View (Listview) sRow - Defaults to Whole Screen sCol - Defaults to Whole Screen Rows - Defaults to Whole Screen Cols - Defaults to Whole Screen KeyNumber - the Key to use when reading the data, used with other parameters Caption$ - the Caption to display Path$ - Alternate path to use for data files (Prepended to the name found in the layout) KeyMatch$ - Show only records that match this key SearchMatch$ - Show only records that contain this substring in them anywhere CallingProgram$ - used for logging purposes, pass in the system function Program$ mat Records - Show only these records mat IncludeCols$ - Show only these columns mat IncludeUI$ - Which UI Options to show mat ColumnDescription$ - Show these captions. If blank, use the descriptions from the layout mat ColumnWidths - Use these widths for the data, if not given, calculate from the file layout mat ColumnForms$ - Display Format for the data, including DATE and FMT and PIC mat DisplayField$ - Field to display on the Loading Window (or REC to show record number or COUNT to show count. Will properly display julian date fields in combination with ColumnForms$ above) mat FilterFields$ mat FilterForm$ mat FilterCompare$ mat FilterCaption$ mat FilterDefaults$ mat FilterKey - Used together to filter the data
fnCSVImport
fnCSVExport
FileIO Add-on Packages
Many FileIO Add-on packages are currently in the works.
ScreenIO Library
The ScreenIO Library is a sister library to the FileIO library. The ScreenIO library requires the FileIO library in order to run.
The ScreenIO library is a complete Rapid Application Design tool that enables you to implement custom screen functions anywhere in your exiting programs.
If you call the ScreenIO Library as a function library, you call a function called fnFM and tell it which screen you wish to use. The ScreenIO library loads your user interface, loads your data files, and preforms all the file maintenance operations you have designed into your screens.
You can find out the latest information about the ScreenIO library in the ScreenIO page.
What’s New
Spring 2009
New Functions
The FileIO Library has been updated in Spring 2009 to provide several new functions:
fnReadRecordWhere fnKey$ fnBuildKey$ fnReadUnopenedDescription$ fnUpdateFile fnDisplayLength fnLength fnReadLayoutHeader fnReadEntireLayout
Speed Increases
Thanks to the BR Profiler, we have increased the speed of FileIO fnOpen function by ten times when running over a LAN and by 100 times when running over the internet using Client Server.
In order to get the new Speed Upgrades, you will need to make sure that you use the latest copy of the fnOpen function in each one of your programs that use FileIO.
Network FileIO
This version of FileIO has been optimized to work better in network situations.
FnSettings
There are more configuration options in the fnSettings routine.
Automatic Update Speed Fix
The automatic update proceedure has been made to run more then 100 times faster then before according to our benchmarking tests.
Fall 2008
DataCrawler Grid
By popular request we added a read/write version to the data crawler. This version works exactly the same as the datacrawler did before but it displays all the contents of your data files in a grid instead of a listview.
When you run the fileIO library as a program, it launches a tool known as the DataCrawler. The DataCrawler shows a listview displaying all your layout files in it. If you select one, it builds another listview with all the data in your selected data file.
If you are looking at the first list of all your file layouts, and you press F5, a grid will be build displaying all the data in your data files. You can change any of the records you like, and when you're done, your changes will be saved to the data file. Additionally, you can Add or Delete records using the buttons at the bottom. Any changes you make are not written to the disk until you click the "Save" button. If you press ESC (Cancel) the grid is closed and all changes you made sinse the last save are lost.
This tool is for programmers only. Do not give your end users access to the DataCrawler.
Use the DataCrawler at your own risk. Gabriel Bakker and Sage AX are not responsible for any harm that comes to your data files through the use of this or any other tools we offer.
The DataCrawler is not designed to be used to maintain your data files. It can be used carefully to correct small things in your data files.
Support for Nonstandard Paths
Many BR Vendors keep different versions of the same data files in different locations. For example, sometimes a BR vendor will use a different Data folder to represent data for different Warehouses, or Customers. In cases like this it is necessary for your programs to specify the path to your data files. They may do so by specifying the optional "PATH" parameter to your data files. See the section #fnOpenFile for more information.
Support for Nonstandard Layout Files Path
Old versions of the fileIO library required you to place all your file layouts in a subfolder of your program directory called "filelay". We have now changed the Fileio library to allow you to place your file layouts any place you want. The only requirement is that they are all together in one folder by themselves.
If you use a nonstandard path in the fileIO library, it is necessary to make a change to the fnSettings code in your copy of fileio.br.
Prompt on Create
The FileIO library automatically creates any data files that it can't find. This was done to make it easier to deploy your finished programs - if you had any empty data files you could omit them from the packages and they would be created when they were needed, on the fly.
The current version of the FileIO library contains the same ability. However, it prompts you before creating any data files. This helps to avoid bugs that happen from incorrectly specifying the path to your data files.
However, if you do intend for your data files to be autocreated, then you probably don't want your end users to modify them. Therefore, you can use the PromptOnCreate setting in the fnSettings code to specify. If this value is set to true, then the fileIO library will prompt you whenever it attempts to Autocreate a data file. If it is set to false, then the fileIO library will create the data files without prompting you, like it always did before.
fnSettings
The newest version of the FileIO library supports some features that require you to specify Global Settings values. These are done by modifying the contents of the fnSettings routine at the beginning of the fileIO library.
Fall 2006
Many improvements have been made in the FileIO library over the summer. This section is intended to acquaint you with the highlights of those changes. Most of these improvements were built from ideas generated during the discussion at the April conference.
Intermixed String and Numeric Specs
The File Library has been expanded to allow the reading of data files that contain mixed string and numeric specs. This is to aid those of you who are planning on implementing the FileIO Library on existing data files which may not be organized with strings first and numbers second.
The central idea of the FileIO Library is based upon the reading of your data files into a String and Numeric array. This will enable you to refer to the fields in your file by using a named subscript, saying F$(FM_NAME) to refer, for example, to the farm file’s name field. The advantage to this is that when you change the file layout, all your existing programs will not have to be modified, because they will only be looking at F$(FM_NAME). If the name field changes from the third string field to the fourth, the value of FM_NAME will also change, and you won’t have to worry about updating your data files.
However, in order to support the reading of a data file with intermixed string and numeric specs, the generated form statement will actually calculate the position of any fields that are not in order, so that the file read statement will still return all string fields first (into your string array) and the numeric fields second (into your numeric array). You do not need to worry about this; the library does it for you. All you have to do is read your file and use the data values.
The only thing that is required is making sure that all your string subscript names end with a “$”. This will tell the library that they are strings. Thank you, George, for this suggestion.
Versioning / Automatic Updates
The file layouts have been expanded to contain a version number. This version number will be used to determine when a file needs to be updated. The version number is the third parameter in the file layout.
Any time you wish to change your file layouts, simply increment this version number. Each time the library attempts to open a data file, the file’s version is checked and compared with the version in the file layout. If the file layout has been changed (if the version in the file layout is greater then the version in the file) then the file will be updated to the latest version. Depending on the file size this may take a couple of moments. A simple progress bar will be displayed on screen while this is happening. The progress bar will be displayed in its own window, so it should not affect anything your programs may have had on screen.
The library uses the following procedure for updating a file. First, the file is copied to a backup file (prefixed by the letter ‘o’ for old). So color.dat would become ocolor.dat, and color.key would become ocolor.key. Then the new file is created and marked with the proper version number. (color.dat, color.key). The old file is opened in read-only mode, and the new file is opened for OutIn. The update routine actually reads through the old file layout, and one record at a time creates a new record, using the new file layout, with the same data, saving it to the new data file.
When it is done upgrading, the progress bar window is closed, and the file is reopened in the fashion you described in your call to fnOpen, and flow returns to your program as though nothing unusual has happened.
If an error occurs during processing, the routine will do what it can to roll your data files back to the previous version, but please make frequent backups of your data files anyway, just to be safe.
Error Checking – If there is a mistake in the file layout
The routine attempts to discover the cause of the error in the case that one is encountered due to a missing file, a file sharing violation, or an invalid or corrupt file layout. If this should happen, the program is paused, and a text message is printed out explaining the most likely source for the error, including what part of the file layout, if any, may have caused the error.
You may then examine the printed text, and the contents of the BR system variables ERR and LINE to determine the problem. If you type “GO”, the next line will be execute “system”, ending your program.
If the file was in the middle of an upgrade when the error happened, it will be automatically rolled back to the previous version, so that when you fix the problem and try to run your program again, the file will again attempt to update from the beginning, and you won’t have to worry about corrupted data.
However, please backup your data before upgrading your data files anyway, just to be safe.
Implementation of Keys / Creation of New Data Files
The library has been expanded to automatically create all new data files, and to automatically update any existing files. This means that in the file layout header, two new fields that were previously ignored are no longer ignored. The RECL value that you specify in your file layout will be used, along with the description/definition of your keys file. When you open a new file (or update an existing one), the library will calculate the proper kps and kln by evaluating the key description. This key description must match up with subscript values from the data elements in your file, and more then one may be specified, but they must be separated with slashes (/). If I wanted my Farm File to be keyed based on CODE and NAME, the proper key description would be:
farm.key, CODE/NAME
The library will then look up the position and length on disk of the CODE and NAME fields and put them together to create the proper keys during an update or creation of a new file.
DataCrawler
Perhaps the most exciting new addition to the library is the addition of a programming tool I like to call a DataCrawler. If you run the FileIO Library directly as a program, instead of using it as a library, it will function as a DataCrawler. The DataCrawler requires a New Gui version of BR, as it requires a ListView to be able to properly and easily display the data in your files.
If you run it in an older version of BR you will get a message, telling you to run it in a newer copy. If you run it in a New Gui version of BR with CONFIG GUI OFF, then GUI will be turned temporarily on for the use of the DataCrawler and then turned off again when you are finished. If you run it in a New Gui version of BR with GUI ON, it will just run normally.
When you run the DataCrawler, first you will see a ListView displaying every file layout it can find in the filelay folder. If you select a file, the DataCrawler will open a large ListView with as many columns as there are fields in the file. The Column Headings will come from the element descriptions in your data files, and the column widths will come from the displayed width of the fields. There will be a row for every record in your data file, and you can resize the column widths, and scroll around the data file to view the raw data of your BR data files on disk.
If you are dealing with a particularly enormous data file (50,000+ records) it can take a moment to populate the ListView with the data in your data file. You may hit ESC to stop loading if you like, and view only the already loaded records.
If you would like to look up a particular record (based only upon the primary key for the data file), you may press F4. This will give you a window which asks you to input the key or partial key. When you press enter, the file will be reloaded, starting at the key you specified and continuing on to the end of the file, or until you press ESC. The records you are looking for should appear at the top of the ListView.
To reset the ListView and look at the contents of the entire file again, simply press F4, and enter a blank (“”) key.
Finally, as with any ListView, you may resort your data by clicking on any of the column headings. Then you can use the slider bar at the right to scroll down to the record you desire.
This is a programming tool and is not designed to be used by an end user. This tool will open your file in read-only mode. It will not allow you to modify the data; I leave that as an exercise for the reader. However, it will update any datafiles you view to the latest version (if they need to be updated) when it opens them, just as any other program that uses the library will do.
Appendix (Examples)
Example.br
00010 ! example.br - This program is an example of for the data reading simplification
00020 ! Copyright April 2006 by Gabriel Bakker
00030 ! Distributed open source as a Christmas gift to brag members
00040 !
00100 execute "config gui off"
01020 DIM form$(1)*255
01030 DIM color$(1)*255,color(1)
01040 DIM colorcat$(1)*255,colorcat(1)
02020 library "fileio": fnopenfile, fnreaddescription$
04000 !
04010 Openfiles: ! Open your files here
04020 let colorfile=fnopen("color",mat color$,mat color,mat form$)
04030 let colorcatfile=fnopen("colorcat",mat colorcat$,mat colorcat,mat form$,1)
06000 ! gosub WriteFiles ! If you want to test this line, make sure to drop flag from 4030
07000 !
07010 MainBit: ! This'un here's tha Main Bit
07030 RESTORE #ColorFile:
07100 ReadNextcolor: ! Read next record
07120 read #ColorFile, using form$(ColorFile) : mat Color$, mat Color eof EndReadColor
07180 PRINT Color$(co_name)&" ("&Color$(co_html)&") is a member of the '";
07190 PRINT trim$(fnReadDescription$(ColorcatFile,cc_Name,Color$(co_category),mat Colorcat$,mat Colorcat,mat form$))&"' category."
07290 goto ReadNextColor
07300 EndReadcolor: ! Finished with Color File
08000 STOP
25000 !
25010 WriteFiles: ! Uncalled routine to demonstrate writing files
25105 let color$(co_code)="GD" !:
let color$(co_Name)="Gold" !:
let color$(co_Category)="YL" !:
let color$(co_html)="FFD700"
25110 write #colorfile, using form$(colorfile): mat color$, mat color
25115 let color$(co_code)="LV" !:
let color$(co_Name)="Lavender" !:
let color$(co_Category)="BL" !:
let color$(co_html)="E6E6FA"
25120 write #colorfile, using form$(colorfile): mat color$, mat color
25125 let color$(co_Code)="OR" !:
let color$(co_Name)="Orange" !:
let color$(co_Category)="YL" !:
let color$(co_html)="FFA500"
25130 write #colorfile, using form$(colorfile): mat color$, mat color
25205 let colorcat$(cc_Code)="YL" !:
let colorcat$(cc_Name)="Yellows" !:
let colorcat$(cc_html)="FFFF00"
25210 write #colorcatfile, using form$(colorcatfile): mat colorcat$,mat colorcat
25215 let colorcat$(cc_Code)="BL" !:
let colorcat$(cc_Name)="Blues" !:
let colorcat$(cc_html)="0000FF"
25220 write #colorcatfile, using form$(colorcatfile): mat colorcat$,mat colorcat
25300 return
40000 !
40010 Open: ! ***** Function to call library openfile and proc subs
40020 def fnOpen(FILENAME$, MAT F$, MAT F, MAT FORM$;INPUTONLY,KEYNUM,___,INDEX)
40025 dim _FileIOSubs$(1)*50
40030 let fnopen=fnopenfile(FILENAME$, MAT F$, MAT F, MAT FORM$,INPUTONLY,KEYNUM,MAT _FileIOSubs$)
40040 for Index=1 to udim(mat _FileIOSubs$) : execute (_FileIOSubs$(Index)) : next Index
40090 fnend
50000 !
60000 Ignore: Continue
Color File Layout
color.dat, CO_, 1
color.key, CODE
recl=127
===================================================
CODE$, Color Code, C 6
NAME$, English Name for Color, V 30
CATEGORY$, General Category of Color, C 6
HTML$, HTML Code for the Color, C 6
ColorCat File Layout colorcat.dat, CC_, 0 colorcat.key, CODE recl=127 =================================================== CODE$, Category Code, C 6 NAME$, English Name for Color, V 30 HTML$, HTML Code for the Color, C 6
Example Layout showing multiple keys (price)
price.dat, PR_, 0 price.key, FARM price.ky2, ITEM price.ky3, FARM/ITEM/GRADE recl=127 =================================================== FARM$, Farm Code (or blank), C 4 ITEM$, Item Code, C 4 GRADE$, Quality, C 4 X, Empty, X 37 PRICE, Default Price, BH 3.2 COST, Default Cost, BH 3.2 XOPRICE, Default Christmas Price, BH 3.2 XOCOST, Default Christmas Cost, BH 3.2 MOPRICE, Default Mothers D Price, BH 3.2 MOCOST, Default Mothers D Cost, BH 3.2 VOPRICE, Default Valentine Price, BH 3.2 VOCOST, Default Valentine Cost, BH 3.2