======================================================================
FINDING YOUR OPTIMUM RTVMCONV - AN EMPIRICAL APPROACH
======================================================================
PRODUCT : R:BASE VERSION : 3.1 or higher
CATEGORY : MEMORY SUBCATEGORY : AREAS & MANAGERS
======================================================================
From William B. Driskell, 6536 20th Ave. N.E., Seattle, WA 98115. Bill
is a marine biologist, a computer consultant, an active participant in
the Seattle-area R:BASE users group, and a frequent contributor to the
R:BASE EXCHANGE.
Each of the following three techniques can improve performance by
optimizing R:BASE's use of memory resources:
<> Modifying your system's memory and disk configurations
<> Modifying your program code to minimize your program's need for memory
<> Modifying your memory allocations by adjusting the RTVMCONV setting
in the DOS environment
The first item is nicely covered in the MEMORY.TXT file that comes with
R:BASE, and the second item was discussed in HOW R:BASE ORGANIZES &
MANAGES MEMORY, PART I in the July/August 1991 R:BASE EXCHANGE.
This article, part II in the memory management series, teaches you how
to use performance indicators and the ISTAT functions to find your
optimum RTVMCONV setting.
Memory Review
=============
First, here's a brief review of R:BASE's memory scheme from the previous
article. R:BASE is loaded into conventional memory in four areas:
<> Static code area
<> Static data area
<> Dynamic code area
<> Dynamic data area
The static areas comprise the actual R:BASE program and some fixed data
and together take up nearly 300K of memory. The dynamic code area and
dynamic data area contain sections of the R:BASE program and buffers of
the database. Each dynamic area gets slices of the remaining memory. In
the metaphor of a workshop, the dynamic data area is workbench for R:BASE
and the dynamic code area is the toolbox. To extend the metaphor, the
static code area contains the power tools and the static data area holds
the stock inventory.
Each dynamic area has a manager program that constantly swaps or discards
working copies of pieces of R:BASE's program code (RBASE.EXE) or
the database's data to use the allocated memory space effectively.
Functionally, the dynamic data manager and dynamic code manager act like
parking lot valets.
The RTVMCONV Setting
====================
Although the static areas are not adjustable, you can adjust the memory
allocation for the dynamic areas because it's controlled by the RTVMCONV
setting in the DOS environment. The default RTVMCONV setting is 640,150
for R:BASE 3.1B and 512,128 for R:BASE 3.1A.
When you start R:BASE, R:BASE checks the current RTVMCONV setting to
determine how much of the remaining memory to allocate to each of the
dynamic areas. For example, if the RTVMCONV setting is 640,150, R:BASE
knows it must reserve 150K of the available memory for the dynamic data
area and then give the rest of the available memory, up to 640K, to the
dynamic code area. Typically, only about 220K of memory is actually
available after the static areas are loaded, so with a setting of 640,150,
the dynamic code manager would get only about 70K. More space reserved
for dynamic data to hold, for example, a lot of complex report expressions
means less space for dynamic code. When less space is available for
dynamic code, more swapping occurs. That takes time, so your R:BASE
applications might run more slowly.
If you don't need so much space for the complex forms or reports
(remember, forms and reports are data in your database), you can reduce
the data setting to 128K by exiting from R:BASE and entering the
following at the DOS prompt with no spaces around the equal sign:
SET RTVMCONV=640,128
The Optimal Allocation
======================
To find the optimal allocation for your application, you need to
understand the trade-offs between performance and complexity. A larger
dynamic data area (set by a larger second number in the RTVMCONV) gives
R:BASE a larger workbench but a smaller toolbox.
With a larger second number, R:BASE can fit a more complex structure,
form, or report on the workbench but will have to do a lot of running to
the store room to get tools that will fit into the smaller toolbox. But
if the data set is large and a complex sorted query is run, the dynamic
data manager needs the larger workbench to handle the data.
In other words, a smaller second number (representing the dynamic data
area) leaves more room for the dynamic code area, thus, R:BASE will
usually be faster because it doesn't have to swap pieces of RBASE.EXE
as often. But the second number (representing the dynamic data area)
must be large enough to run your application without running out of
memory.
Because the total dynamic memory allocation is a rob-Peter-to-pay-Paul
situation, anticipating the optimal memory balance might seem virtually
impossible. But let's try an empirical solution to this dilemma.
The Empirical Timing Approach
=============================
To optimize a single application, you can empirically adjust the RTVMCONV
value simply by using execution time as a performance indicator. That is,
run your program noting the execution time, exit to DOS, adjust the
RTVMCONV setting, and then rerun the program. After several trials, you
will be able to fine-tune your RTVMCONV to minimize execution time. If
you set the value for data allocation (the second number) too low, your
program will stop with Out of dynamic space or Out of memory error
messages. If you set it too high, the dynamic code manager is
constrained by churning code through the limited space, severely
degrading performance.
By setting up time variables at the beginning and end of a routine with
R:BASE's #TIME system variable, you can calculate the elapsed time of
execution. This process is appropriate only for command files and
reports, not for interactive forms. Here's an example:
SET VAR vbegin TIME = .#TIME
*( ...)
*( Your R:BASE program goes here.)
*( ...)
SET VAR vstop TIME = .#TIME
WRITE .vbegin, .vstop
Using a small command file, I was able to reap a 25% time savings
resulting from a reduction in the dynamic data area's allocation from
the 150K default to 120K.
The routine ran out of memory when the value reached 80K. For the sake
of future data expansion, I finally used 120 rather than 90 for the
optimal dynamic data area setting (the second number).
This type of optimization can be particularly useful for large
applications that seem to grind on for hours. Even a small difference
might yield significant time savings.
Using ISTAT to Measure Memory Use
=================================
Using a form to enter or edit data is an interactive process, so the
empirical timing technique would be imprecise in assessing the form's
performance. But in some cases, the technique can still be valuable in
terms of timing an overall task. An alternative approach, discussed below,
assesses the actual memory demands by using ISTAT to get rough indicators.
The ISTAT() Functions
=====================
The ISTAT functions were introduced in R:BASE 3.1B as a way to check
database size, the amount of disk space remaining, and memory status.
Database size and disk space are self explanatory; memory status is a
bit more complex.
ISTAT can return four status checks on conventional memory:
<> ISTAT('MEMORY') the amount of free memory (in bytes) remaining to
the operating system.
<> ISTAT('TOTALALLOC') the amount of memory allocated for the dynamic
data area that has been used so far in the session. This value might
be less than the second number in the RTMVCONV setting if the data
manager hasn't requested the maximum reserved.
<> ISTAT('TOTALFREE') the amount of memory free inside the current
dynamic data area.
<> ISTAT('MAXFREE') the size of the largest single memory block free
inside the current dynamic data area.
You can get a quick snapshot of the disk, the database, and your system's
memory by combining all the ISTAT functions into a single command file
like this one:
*( ISTATS.CMD--diagnostic routine to show memory and disk status.)
CLS FROM 6
SET VAR vi INTEGER = (ISTAT('DISKSPACE'))
WRITE .vi AT 6,37 USING 'Free disk space: 99,999,999 bytes'
SET VAR vi = (ISTAT('DBSIZE'))
WRITE .vi AT 8,39 USING 'Database size: 99,999,999 bytes'
SET VAR vi = (ISTAT('MEMORY'))
WRITE .vi AT 10,30 USING 'DOS free memory remaining: 999,999 bytes'
SET VAR vi = (ISTAT('TOTALALLOC'))
WRITE .vi AT 12,12 USING 'Total dynamic data area+
currently allocated: 999,999 bytes'
SET VAR vi = (ISTAT('TOTALFREE'))
WRITE .vi AT 14,21 USING 'Total dynamic data area free space:+
999,999 bytes'
SET VAR vi = (ISTAT('MAXFREE'))
WRITE .vi AT 16,5 USING 'Largest contiguous free space in +
dynamic data area: 999,999 bytes'
CLEAR VAR vi
RETURN
Actual Memory Allocated Approach
================================
You can track the actual memory demands of an application, even one that
uses interactive forms, by using expressions to hold the maximum amount
of memory that was allocated during execution. Initialize vmax to an
INTEGER and set it to zero at the top of the application:
SET VAR vmax INTEGER = 0
Then use the following expressions in the application and in forms and
reports to track the maximum allocation:
valloc = (ISTAT('TOTALLOC'))
vmax = (IFGT(.vmax,.valloc,.vmax,.valloc))
After running the application, look at the value of vmax to see the final
maximum memory that was allocated during the routine.
In a command file, you don't need to constantly check and store the value.
Display the final ISTAT value at the end of the routine:
SET VAR valloc = (ISTAT('TOTALLOC'))
WRITE .valloc
Once you know the maximum allocation demand in bytes, you can use it as
a guiding value (it might be slightly inflated) for resetting RTVMCONV.
Exit R:BASE, then reset the second value in the RTVMCONV setting to the
actual maximum allocation value of the application in K bytes. (To convert
from bytes to K bytes, divide the byte total by 1024 and round up.)
Then restart R:BASE and rerun the application. This technique will
probably result in a performance gain because you are returning unused
data memory to the dynamic code manager. That is, you're taking unused
workbench space and giving it to the toolbox. If the allocation is set
too low, the routine will crash and R:BASE will display an Out of dynamic
space error message.
Unfortunately, the maximum allocation demands might not precisely reflect
the dynamic data needs. This is because the dynamic data manager allocates
memory in blocks of 64K whenever possible and then smaller chunks only
when a full 64K block is not available. Therefore, vmax might show that
131,036 bytes (128K bytes) were allocated, but the total block might not
have been used.
The optimal RTVMCONV data setting probably lies somewhere in the second
block between 65K and 128K. When calculating the K bytes from the ISTAT
value, I usually round up to the next 10K since a 9K overhead appears to
be hidden and not reflected in the ISTAT value. For example, ISTAT reports
a 90K RTVMCONV data setting (640,90) as a 82,862-byte allocation.
You can get an even tighter memory snapshot of a single line of code by
calculating the actual amount of memory in use at that particular moment.
The following example assumes that you have already initialized Microrim
Technical Notes vmaxin as an INTEGER equal to zero at the top of the
application.
-- use of dynamic data memory
SET VAR vfree INTEGER = (ISTAT('TOTALFREE'))
SET VAR valloc INTEGER = (ISTAT('TOTALLOC'))
SET VAR vinuse = (.alloc - .free)
-- option to store max value
SET VAR vmaxin = (IFGT(.maxin,.inuse,.maxin,.inuse))
Place this code right after the most complex, memory-intensive section
of the routine being assessed so you don't get some misleading mid-process
or finishing statistics that might not reflect the maximum usage.
ZIP Effects
===========
The ZIP command causes the dynamic code manager to release most of its
allocated memory in order to load and execute a DOS program. As a result,
upon returning to R:BASE, the ISTAT functions reflect the much-reduced
memory allocation. At this point, ISTAT(TOTALALLOC) is actually showing
only the current usage, not the available allocation. As more complex
(more memory-hungry) commands are run, ISTAT(TOTALALLOC) increases as a
result of the dynamic code manager's requests for larger allocations.
The ZIP ROLLOUT and ZIP RETURN commands both show a similar effect by
releasing both dynamic and static memory areas. In this case, R:BASE
copies its internal environment into a .$$$ archive file and then shells
out to DOS. Conventional memory retains only a kernel of the R:BASE static
code about 15K versus the full 295K. The 15K is just enough to reload the
environment and restart the session. After returning to R:BASE, the
ISTAT(TOTALALLOC) value is again minimal but increases as more complex
operations are executed.
Forcing the Data Manager
========================
Under certain circumstances, the data manager might be unable to free up
enough space to accomplish an operation. For example, during a session in
an accounting application when a user switches tasks from data entry to
reports on benefits, the data manager might need to remove more from its
buffers than normal operations generally request; the data manager might
not be able to find the space to release, so it returns an error message
instead. Under these circumstances, the user can force the data manager
to clear its buffers by disconnecting and then reconnecting the database
or by using the ZIP ROLLOUT or ZIP RETURN commands.
Other Optimizing Techniques
===========================
It's the function of the dynamic code manager to shuffle pages of data
and code into and out of memory. Therefore, the configuration of extended
or expanded memory and the use of memory management utilities, RAM disks,
and disk caches can have significant effects on R:BASE performance. Refer
to the memory.txt file on your latest R:BASE installation disks for
Microrim's latest recommendations.