February - 2017

Since the very first caving expedition to Mulu in 1978 new cave discoveries have been surveyed and published in the expedition reports.  Over the years the totals of surveyed cave has been collated and by the end of the 2011 expedition over 362km of passage had been recorded within the Gunung Mulu National Park.  During this 32 year period, methods of data management have advanced immeasurably, from discarded boxes of survey notes to modern computerised survey management systems.  Managing this lengthy process has been beset with historical and technical difficulties which have no doubt been mirrored in other caving areas of the world.  This is the story of how the Mulu Caves Project has overcome the problems to produce, what we believe to be, one of the finest large scale datasets of historical cave survey data in the world.

Data collection in the 1980's. Photo © Jerry Wooldridge

In the early days, when slide rules, cosine tables and Chartwell semi-waterproof note books were the tools of the cave surveyor’s trade, databases were mostly unheard of.  The early Mulu expedition cavers went about their business with the best intentions.  Notes were taken underground, field drawings produced on graph paper and finished inked surveys drafted on return to the UK.  These were then published in the expedition report and that was it, an independent expedition completed until the next time.  No great value was given to the raw data at the time.  Survey notes were held by various members, stored in garages and attics until some became lost.  On the 1978 expedition they had so much to survey that it is rumoured that the surveyors recycled the waterproof paper, wiping it clean once the data had been processed!  It is very unfortunate that the raw data from the late seventies and early eighties, nearly one hundred kilometres of survey, was mostly lost.

It was in the mid to late nineties that the first meaningful attempt at building a Mulu dataset got underway.  Dave Gill was employed at the National Park as the development officer and had a major influence on the expeditions of that decade.  He began to contact all the key members of the early expeditions and pull together what data was still available.  This period coincided with the development of computerised survey programmes and Dave eventually settled on the Compass software developed by the American, Larry Fish.  All the data from the 1988 expedition onwards was readily obtainable.  Certain discoveries, such as Cobweb Cave, were available from the 1984 expedition but very little remained from 1978 & 1980.  To make matters worse Dave lost his job and was ejected from the country in 2000 and his well collected dataset suffered as a result.

2003 saw the start of the fledgling idea to pull all the independent expeditions under one banner – the Mulu Caves Project.  This was to be driven by veterans Matt Kirby, Dick Willis and Tim Allen.  It was also the year that Wookey, a caver with serious cave survey computing skills, joined the expedition.  Wookey’s preference for data processing and visualisation software was Survex, as it was very good for the sort of multi cave dataset we had in mind for Mulu.  He persuaded the team to adopt the programme, although we knew he was a little biased as he was heavily involved in the development of the software.  During the course of the expedition Dave Gill’s Compass files where converted into Survex.  Some further work was also done after the expedition returned to the UK as a case-insensitivity between the two programmes needed fixing.

The 2003 expedition focused its attention on Gunung Benerat and was eager to complete as much of the dataset for this mountain as possible.  All the real data that existed had been converted to Survex but that still left caves such as Blue Moonlight Bay, Terikan River Cave, Benerat Caverns, Sakai’s and Tiger cave with either part data or no data at all.  We wanted to be able to visualise all the caves on the computer so that we could view them in relation to one another.  This would help to better understand the geology and help us to find the connections.  Fortunately the surveys in the expedition reports were high quality and contained both plans and elevations.  This enabled us to reconstruct fairly plausible data using a method designed by Wookey (link to method), which created a set of three dimensional coordinates which were then entered into the dataset as if they were the raw data.  By the end of the expedition a fairly comprehensive survey of all the known caves in Benerat was available.

2005 saw the rapid expansion of Whiterock Cave across the northern part of Gunung Api and its inevitable connection into the Clearwater System.  Following that expedition a huge amount of work took place to bring the Gunung Api dataset up to, and beyond, the standard of Benerat.  The Whiterock data had been entered as the cave was surveyed but in a some what unstructured manner.  During the winter of 2005/6 Matt Kirby, who was new to Survex, decided to learn to use the system by re-entering the recently entered Whiterock data. Matt decided to standardise the way in which data was entered and stored in order to simplify the existing model and make it as user-friendly as possible. This resulted in a new Survex model for Whiterock. With the new structure for Whiterock in place Matt proceeded to enter all the available raw data from expeditions as far back as 1988, to create a model for the whole mountain. This include Blackrock and the Armistice series of Clearwater together with the 1996/98 discoveries in the Hidden Valley.  Unfortunately data from expeditions pre ’88 was not available so Tim Allen went through the laborious process of reconstructing the missing data from the original surveys.  This included some fifty kilometers of the original Clearwater survey together with Lagans Cave, Cobra cave and Nasib Bagus.  This set the format for the current Mulu dataset.

With a working model in place for the Api side of the river Matt decided to re-work the Benarat dataset to bring the structure in line with that of Api. This was a laborious task as the data had been entered by various people over the course of several expeditions which had resulted in some complex links and commands that would make interrogation very complicated at a later date. When the Benarat data had been re-worked, the two datasets were joined together to produce the Mulu Dataset we use today.

The Dataset Structure

The dataset has been structured to conform to a strict hierarchy of .svx files on six levels. The files are stored in folders (shown yellow) which follow a similar hierarchy apart from the bottom level where raw data .svx files and the cave master .svx files are both stored in the same folder. This ensures that all the information required to produce a 3D image of a cave is in the same location. Mountain Master files, Mountain Wireframe files and Mountain Fixed Point files share the same folder.

The Dataset Hierarchy

A brief description of each level is given below, starting at the lowest level in the structure:-

Raw Data Level

All the raw survey data is at the lowest level in the structure. All levels above this are simply used to include and link subordinate data. Raw data files include all the information from surveying trips.

Cave Master Level

The Cave Master level is a .svx file which tells Survex which subordinate raw data files (in the cave folder) to include and how to link them together using the ‘equate’ command. Where a GPS fixed point is used to position a cave the ‘fix’ command appears in this file.

Area Master Level

The structure is laid out to allow connected groups of caves to be viewed together either as a small group at the ‘Area’ level or the whole system at the ‘System’ level. The Area Master level is a .svx file which tells Survex which Cave Master .svx files to include in the area and how to link them together. This level is always used but in some cases, where the system is uncomplicated, there is only one area file within a system. The Area folders are used to break systems into smaller parts for ease of viewing. This has been very useful to break the Clearwater System into the Clearwater end and the Blackrock/Whiterock end. The area files overcome the need to open the Cave System files when only a small portion of data is required.

System Master Level

The System Master level allows all caves belonging to a particular cave system to be viewed as one. This level is a .svx file which tells Survex which subordinate Area Master .svx files to include in the System and how to link them. This overcomes the need to open the whole dataset when a particular system is to be viewed.

Mountain Master Level

The Mountain Master level is a .svx file which tells Survex which System Master .svx files to include and how to link them together. This allows each mountain to be viewed without having to open the full dataset. Three Mountains are included Benarat, Api and the Southern Hills.  Mountain fixed point files and Wireframe files share the same Mountain folder.

Mulu Master Level

The Mulu Master level is at the top of the tree. This level is a .svx file which tells Survex which Mountain Master .svx files to include and how to link them together.

The WireFrame

It was a fantastic achievement to be able to visualise all the Mulu caves in 3D and in relation to one another.  It would be even better to be able to visualise the caves in relation to the mountains as well.  Thus the idea of constructing a surface topography connected to Survex was born.  A number of methods were considered where the surface topography could be represented by Google Earth or NASA imagery, but these were found to be either too technical to display or unsuitable for our needs.  In 2006, Tim Allen built a simple wireframe using a 250m grid for the area covering the Southern Hills, Gunung Api and Gunung Benerat.  Spot heights were taken from the 1:50,000 topographical map and the grid converted to a Survex file and incorporated into the dataset.  This was a laborious task which required many hours of painstaking work but allowed the wireframe to be viewed in its entirety or split into the three major hills.  At this time Survex data could only be viewed as either ‘cave’ or ‘surface survey’ and a new image type was required for the terrain data.   Eventually, Phil Underwood of the Survex Group provided an amended version of Survex with a third option for surface terrain.

Survex line plot looking from the west showing all Mulu caves and the wireframe.

Between the 2007 and 2009 expeditions Tim Allen and Matt Kirby continued to update the Survex dataset.  All the available survey data for the Southern Hills was added and, where this was not available, reconstructed data was generated from the original surveys.  At the Hidden Earth Conference in 2008 the Mulu Caves Project was awarded the Auther Butcher Award for excellence in the field of cave survey for the work with the dataset and wireframe.

There is still a little work to do to complete the task.  Extensions made to Racer cave and Stonehorse cave by another group in the 1990’s was not drawn up to a good enough standard to reconstruct comparable data from.  These will probably require resurvey.  In 2010 a small expedition to the Southern Hills tied all the known caves together with a surface survey which improved the dataset for that area. The 2011 expedition carried out a radiolocation exercise near Camp 5 which allowed the cave passages near the Melinau Gorge to be accurately positioned relative to the surface features. This will assist on the search for a northern entrance to the Clearwater System which has, to date, proved elusive.

It is now a simple task for any future expedition to add their discoveries and continue to expand the dataset.  The dataset requires an element of control otherwise differing versions would soon appear.  The Mulu Caves Project master copy is maintained by Matt Kirby.