""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
HOW R:BASE ORGANIZES & MANAGES MEMORY -- PART I
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
PRODUCT : R:BASE VERSION : 3.1
CATEGORY : MEMORY SUBCATEGORY : AREAS & MANAGERS
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Memory management is a complex, technical subject. Here's a basic
introduction to how R:BASE lays out and organizes your computer's
conventional memory.
Future R:BASE EXCHANGE articles will go into more detail about what
actually happens in these areas. The purpose of this article is to
give you the overall picture of the four memory areas and the two
dynamic memory managers. This lays the groundwork for future articles
that will show you how to use this knowledge to improve performance.
R:BASE Memory Areas
"""""""""""""""""""
R:BASE has four memory areas--two dynamic and two static.
o Dynamic Data Area--holds memory blocks, each with its own memory
handle. Memory blocks hold pieces of R:BASE commands,
applications, and databases. If R:BASE says you've run out of
dynamic memory or buffer space, it's your Dynamic Data Area that
has run out of room.
o Dynamic Code Area--holds virtual memory pages containing pieces
of R:BASE internal code. Each virtual memory page is a little
piece of the RBASE.EXE program.
o Static Data Area--holds some types of data that go into memory
and stay put. R:BASE handles this area internally. It always
occupies 100K of memory.
o Static Code Area--holds input and output code that goes into
memory and stays put. R:BASE handles this area internally. It
always occupies 200K of memory.
R:BASE Memory Map
"""""""""""""""""
Here's a picture of how conventional memory might look if your memory
has no TSRs (terminate and stay resident programs) and uses exactly
70K for network drivers.
Dynamic Memory Managers
"""""""""""""""""""""""
The two dynamic memory managers manage the dynamic memory areas.
o Dynamic Code Manager--moves pieces of R:BASE internal code (code
from RBASE.EXE) in and out of the Dynamic Code Area. As it does
this, it tries to keep the most frequently used code in the
Dynamic Code Area to make R:BASE faster. The Code Manager comes
from ".RTLink" from Pocketsoft, Inc.
o Dynamic Data Manager--moves R:BASE application programs,
variables, and database information in and out of the Dynamic
Data Area.
How DOS Loads an .EXE
""""""""""""""""""""""
DOS loads an .EXE program into memory in four steps:
o It opens the .EXE file and reads the .EXE header. The header
gives DOS the memory requirements.
o It allocates the amount of memory that the program needs.
o It reads the program into memory.
o It corrects (fixes) the program's relative memory addresses. A
relative memory address is a memory location relative to the
start of the program. Once DOS knows the absolute address of the
start of the program, it converts (fixes) the relative addresses
into absolute addresses.
Loading RBASE.EXE
"""""""""""""""""
When DOS loads RBASE.EXE into memory, it allocates space for the
static areas, and loads the two dynamic memory managers into the
static code area.
Next, R:BASE initializes the Dynamic Code Manager, which allocates
memory for the Dynamic Code Area.
Then R:BASE initializes the Dynamic Data Manager, which allocates
memory for the Dynamic Data Area. Now R:BASE is ready to go to work.
Static Data & Static Code Areas
"""""""""""""""""""""""""""""""
The Static Data Area holds items that because of frequent use must be
fast, items that can't go in the Dynamic Data Area, and items that are
a constant size and must always be kept in memory. Here are some
examples:
o Keyboard buffer
o R:BASE settings
o The stack--all DOS programs have a stack to control flow and
store temporary values
o Database information such as the database path and name
o Virtual Page information table
o Memory block information table
The Static Code Area holds input and output code like this:
o Screen output code
o Keyboard input code
o File input and output code
o Database input and output code
o Dynamic Code Manager
Dynamic Code--Virtual Pages
"""""""""""""""""""""""""""
When DOS finished loading R:BASE it really only loaded a small part of
the R:BASE code. R:BASE tricks DOS into thinking that it contains 200K
of code and 100K of data. Actually it contains about 1800K of code.
MICRORIM ONLINE July 1991 ------------------------------ Page 10 of 34
The .EXE header was modified to prevent DOS from trying to load the
whole program and returning an out-of-memory error because it was too
big to fit in 640K of memory.
The remaining 1600K of code is broken up into 950 individual pieces,
called "virtual pages." As the virtual pages are needed, the Dynamic
Code Manager reads them from the .EXE into the Dynamic Code Area and
executes them there.
Each virtual page contains its own code and relative memory reference
table, so each page is like a separate .EXE. This allows each to be
moved around in memory. Each time they're moved into memory, the
Dynamic Code Manager fixes the relative memory references into the
appropriate new absolute memory references in order to execute that
code in the new absolute memory location.
When the code in a virtual page needs to be executed, it's the Dynamic
Code Manager's job to load that page and execute the desired code.
When the Dynamic Code Area is full and another page needs to be
loaded, the Dynamic Code Manager decides which virtual page to discard
and where to put it.
The Dynamic Code Manager keeps the most recently and most heavily used
virtual pages by discarding the least recently used page. It knows
which one this is because it increments a counter each time a page is
used, and decrements that counter over time. It discards the pages
with the smallest counter values.
The Dynamic Code Manager decides where to put the discarded page.
First choice is to swap it to expanded memory (EMS version 3.2 or
higher). Second choice is to swap it to extended memory (XMS version
2.0 or higher). If it can't do either of these, it discards the page,
re-reading the page from disk if it's needed again.
Virtual pages are like little .EXEs. They're movable, swapable, and
discardable. The location of each virtual page is stored in the Static
Data Area, so the Dynamic Code Manager can quickly find each one.
Dynamic Data--Memory Blocks
"""""""""""""""""""""""""""
After the Dynamic Code Manager allocates memory for the Dynamic Code
Area, the Dynamic Data Manager allocates memory for the Dynamic Data
Area.
The Dynamic Data Area is made up of memory blocks each with its own
identifying memory handle. There can be a maximum of 300 memory blocks
(handles). The Dynamic Data Area has both dynamic memory space (data
buffer or data space) and up to 300 memory handles (blocks). You need
to be careful not to run out of either one.
Here are examples of the dynamic data that R:BASE puts into memory
blocks in the Dynamic Data Area:
o Table and column lists
o Database, file, and sort buffers
o R:BASE commands and programs
o R:BASE global variables
The memory blocks that make up the Dynamic Data Area can vary in size
and move around in memory. They can be swapped to disk or even
discarded. The Dynamic Data Manager knows the following about each
memory block in the Dynamic Data Area:
o Its size
o Its location in memory or on disk
o If it's in use
o Whether it can be swapped to disk
o If it can be discarded
A database buffer is an example of a memory block that can be
discarded when it isn't being used. This is because the data in a
database buffer is just a copy of the data in the database. It's
faster to just throw it away and re-read it from the database than to
swap it to disk and re-read it from the swap file.
When R:BASE needs a new memory block, the Dynamic Data Manager must
decide where in the Dynamic Data Area to put the new block. First, it
looks for unused memory; if it finds enough, it's done. If it doesn't,
R:BASE looks for an existing memory block that can be swapped or
discarded. If it can't find a single block that will provide enough
space, it swaps or discards all the blocks. If there still isn't
enough free memory, R:BASE gives an out-of-memory error.
A smaller Dynamic Data Area leaves more room for the Dynamic Code
Area, so R:BASE won't have to swap the virtual pages as much. But the
Dynamic Data Area must be large enough to run your application without
running out of memory space.
The Dynamic Memory Area Sizes
"""""""""""""""""""""""""""""
The dynamic areas vary in size depending on the amount of free memory
available after loading the static areas, and the value of a DOS
environment variable (RTVMCONV), which you can set before you start
R:BASE. A future article will explain how to set RTVMCONV and how to
use the ISTAT function to check memory.
The size of the Dynamic Data Area limits the complexity of
applications, the maximum size of the database schema, and the
performance of complex applications.
The size of the Dynamic Code Area impacts the performance of R:BASE in
general and limits the number of R:BASE features that can be used
together, and still run quickly.
Tips for Better Performance
"""""""""""""""""""""""""""
Here are some performance enhancement tips:
o Design your code in small chunks so your application will do an
operation only if the user asks for it.
o Avoid comments, long table names, and long column names. They use
up space in the Dynamic Data Area.
o Make your EEPs (Entry/Exit Procedures) in forms small. A large
EEP may cause the Dynamic Code Manager to discard the FORMS
internal R:BASE code, which must then be reloaded when the EEP
finishes.
o Group commands together so R:BASE can keep a particular virtual
page in memory. For example, put all the SET VAR commands
together, all the SELECT commands together, and all the UPDATE
commands together. If you do this, the Dynamic Code Manager won't
have to swap the virtual pages that run those commands in and out
of the Dynamic Code Area. Less swapping and discarding can make
an application faster.
o Make the Browse/Edit menu run faster by setting LAYOUT OFF when
you don't need to save the layout.
o Speed up the R:BASE Main Menu by getting rid of forms and reports
you don't need.
Fewer Out-of-memory Errors
""""""""""""""""""""""""""
When you run out of memory, you have either run out of space in the
Dynamic Data Area, or you have attempted to exceed the maximum number
of memory handles available in the Dynamic Data Area. R:BASE 3.1A
gives you two separate messages so you can know which one you ran out
of (handles or dynamic space). Earlier R:BASE versions say only that
you have run out of dynamic space when in fact you may have plenty of
space but no more memory handles.
Try the following methods to get fewer out-of-memory errors:
o Use less functionality at a single moment in time.
o Remove long names and comments. They take up space.
o Lookups in forms and reports don't take up a lot of space, but
they do use a lot of memory handles.
o Reduce the number of saved lookups in forms by setting LOOKUP to
a lower number. Each saved lookup in a form uses space in the
Dynamic Data Area. By reducing the number, you can reduce a
form's memory requirements. The LOOKUP setting tells R:BASE how
many form lookups to save in memory. Saving lookups in memory
enables R:BASE to display data more quickly in a form, but each
saved lookups uses 500 bytes of memory.
o Don't use as many rules per table. Rules don't use a lot of
space, but they do use up memory handles. For example, you might
exceed the 300 memory handle limit by defining 50 rules on a
single table.
o Reduce the number of columns in a single table. Large tables that
have more than 100 columns can eat up the space in your Dynamic
Data Area. Each table access uses 98 bytes per column of the
Dynamic Data Area.
By keeping tables small and related to other tables, you'll naturally
have fewer rules on a single table. That's why a good relational
design with several related tables--each with only a few columns--is a
much better use of memory. Better use of memory can pay off in better
performance.