How does AEGIS read map cell data? (gat/rsw)

[Kenpachi 65]

Hi there. Long time no see. 😊

I'm currently working on a small server/client application just as a finger exercise for me. Therefor I'm using RO files and stumbled over the .gat cell types and their relation with the water level in .rsw files.

According to *Athena and some other projects there should be just a few possible types:

Spoiler

// from Hercules\src\map\map.c - map_gat2cell()
case 0: cell.walkable = 1; cell.shootable = 1; cell.water = 0; break; // walkable ground
case 1: cell.walkable = 0; cell.shootable = 0; cell.water = 0; break; // non-walkable ground
case 2: cell.walkable = 1; cell.shootable = 1; cell.water = 0; break; // ???
case 3: cell.walkable = 1; cell.shootable = 1; cell.water = 1; break; // walkable water
case 4: cell.walkable = 1; cell.shootable = 1; cell.water = 0; break; // ???
case 5: cell.walkable = 0; cell.shootable = 1; cell.water = 0; break; // gap (snipable)
case 6: cell.walkable = 1; cell.shootable = 1; cell.water = 0; break; // ???

// from Hercules\src\map\map.c - map_cell2gat
if( cell.walkable == 1 && cell.shootable == 1 && cell.water == 0 ) return 0;
if( cell.walkable == 0 && cell.shootable == 0 && cell.water == 0 ) return 1;
if( cell.walkable == 1 && cell.shootable == 1 && cell.water == 1 ) return 3;
if( cell.walkable == 0 && cell.shootable == 1 && cell.water == 0 ) return 5;

 

 

Additionally I found different ways to check if a cell is under water or not:

Spoiler

// *Athena and others
if(gat_cell.upperLeftHeight > WATER_LEVEL_FROM_RSW)
      // cell is under water
else
      // cell is not under water

// OpenKore and others
float averageDepth = (gat_block.upperLeftHeight + gat_block.upperRightHeight + gat_block.lowerLeftHeight + gat_block.lowerRightHeight) / 4;
if(averageDepth > WATER_LEVEL_FROM_RSW)
      // cell is under water
else
      // cell is not under water

// Me for testing purposes
if(gat_cell.upperLeftHeight > WATER_LEVEL_FROM_RSW || gat_cell.upperRightHeight > WATER_LEVEL_FROM_RSW ||
   gat_cell.lowerLeftHeight > WATER_LEVEL_FROM_RSW || gat_cell.lowerRightHeight > WATER_LEVEL_FROM_RSW)
      // cell is under water
else
      // cell is not under water

 

 

So I did a little testing a wrote a tool to collect all combinations of cell types and water level checks in every map. This is the result:

 

What? Now I really got confused. Even if only the *Athena way (IsUnderWaterUpperLeft) to check cell types  is payed attention to, there is a significant difference to what one would expect:

 

See? Maybe I'm using wrong algorithms, but they seem to be correct:

Spoiler

private void ReadFile(string filePath)
{
	_currentFileNumber++;
	_currentFile = filePath;
	_worker.ReportProgress(default);
	var waterLevel = GetWaterLevel(filePath.Replace(".gat", ".rsw"));
	using var reader = new BinaryReader(File.OpenRead(filePath));
	reader.ReadBytes(6); // Skip magic header
	var width = reader.ReadInt32();
	var height = reader.ReadInt32();
	for (var y = 0; y < height; y++)
	{
		for (var x = 0; x < width; x++)
		{
			var ul = reader.ReadSingle();
			var ur = reader.ReadSingle();
			var ll = reader.ReadSingle();
			var lr = reader.ReadSingle();
			var type = reader.ReadByte();
			reader.ReadBytes(3); // Skip unknown bytes
			var cellType = new CellType {Type = type, MapData = $"{Path.GetFileName(filePath),-15} [x={x + 1} - y={y + 1}]", Count = 1};

			if (waterLevel == 0)
				cellType.IsUnderWaterAny = cellType.IsUnderWaterAverage = cellType.IsUnderWaterUpperLeft = false;
			else
			{
				cellType.IsUnderWaterAny = ul > waterLevel || ur > waterLevel || ll > waterLevel || lr > waterLevel;
				cellType.IsUnderWaterAverage = (ul + ur + ll + lr) / 4 > waterLevel;
				cellType.IsUnderWaterUpperLeft = ul > waterLevel;
			}
			
			bool Match(CellType o) => o.Type == cellType.Type && o.IsUnderWaterAny == cellType.IsUnderWaterAny &&
									  o.IsUnderWaterAverage == cellType.IsUnderWaterAverage &&
									  o.IsUnderWaterUpperLeft == cellType.IsUnderWaterUpperLeft;
			if (!_types.Exists(Match))
				_types.Add(cellType);
			else
				_types[_types.FindIndex(Match)].Count++;
		}
	}
}

private static float GetWaterLevel(string filePath)
{
	if (!File.Exists(filePath)) return 0;
	using var reader = new BinaryReader(File.OpenRead(filePath));
	reader.ReadBytes(4); // Skip magic header
	var majorVersion = reader.ReadByte();
	var minorVersion = reader.ReadByte();
	reader.ReadBytes((majorVersion > 1 || (majorVersion == 1 && minorVersion >= 4)) ? 160 : 120); // Skip unused bytes
	return reader.ReadSingle();
}

 

 

So could anybody please tell me, how AEGIS handles this? Maybe Hercules can profit by this, too. 🙄

 

~Kenpachi

[Asheraf 123]

The athena way is what aegis uses, it reads the 4th dword in the cell info struct and compares it with the map water level read from the RSW, the code roughly looks like this (quick draft so i apologize if it's not so clear).

{
	std::ifstream gat_fs(filename, std::ios::binary);

	VALIDATE_MAGIC(gat_fs, "GRAT", 4);
	gat_fs.read(reinterpret_cast<char *>(&m_verMajor), sizeof(char));
	gat_fs.read(reinterpret_cast<char *>(&m_verMinor), sizeof(char));
	gat_fs.read(reinterpret_cast<char *>(&m_width),  sizeof(int));
	gat_fs.read(reinterpret_cast<char *>(&m_height), sizeof(int));

	m_cells.resize(m_width * m_height);
	gat_fs.read(reinterpret_cast<char *>(m_cells.data()), m_cells.size());

	std::for_each(m_cells.begin(), m_cells.end(), [idx = 0](struct CAttrCell &cell) mutable {
		if (cell.flag == 1 || cell.flag == 5)
			m_TileInfo[idx] |= SVR_CELL_BLOCK;
		if (cell.flag != 1)
			m_TileInfo[idx] |= SVR_CELL_ARROW;
		if (cell.h1 > m_waterLevel) // m_waterLevel from RSW
			m_TileInfo[idx] |= SVR_CELL_WATER;
		++idx;
	});
	return 0;
}

 

[Kenpachi 65]

Okay, before reading your post I did some improvements to my code.

At first I noticed that return 0 if no RSW file was found and then ignoring it was stupid, sind 0 is an assumable value.

So I changed it to float.NaN which brought slightly different results:

Spoiler

 

After this I thought about bad rounding of float values when comparing, so I changed the comparison to byte level and got really different results:

Spoiler

 

So I'll stick to the byte level comparison, since this should be more accurate.

Now that I've read your post I noticed, that AEGIS uses the 4th DWORD, which noone else does.

Hercules uses the 1st DWORD:  (Woops! Possible improvement detected. 😋)

Spoiler

// FROM map.c - map_readgat(struct map_data *m)

	// Set cell properties
	off = 14;
	for( xy = 0; xy < num_cells; ++xy )
	{
		// read cell data
		float height = *(float*)( gat + off      );
		uint32 type = *(uint32*)( gat + off + 16 );
		off += 20;

		if( type == 0 && water_height != NO_WATER && height > water_height )
			type = 3; // Cell is 0 (walkable) but under water level, set to 3 (walkable water)

		m->cell[xy] = map->gat2cell(type);
	}

 

 

Let's compare those two:

Spoiler

 

Okay, there's a huge difference. For a better overview only the AEGIS based results:

Spoiler

 

Still not what one would expect, since type 3 should be water.

Let's see what happens when checking for value is less than water level:

Spoiler

 

Well, type 3 looks better now but overall it still looks bad. Let's include equality when comparing:

Spoiler

 

Still... no perfection at all. 😥

My currently used algorithms:

Spoiler

private void ReadFile(string filePath)
{
	_currentFileNumber++;
	_currentFile = filePath;
	_worker.ReportProgress(default);
	var waterLevel = GetWaterLevel(filePath.Replace(".gat", ".rsw"));
	if (waterLevel == null) return;
	using var reader = new BinaryReader(File.OpenRead(filePath));
	reader.ReadBytes(6); // Skip magic header
	var width = reader.ReadInt32();
	var height = reader.ReadInt32();
	for (var y = 0; y < height; y++)
	{
		for (var x = 0; x < width; x++)
		{
			reader.ReadBytes(12); // Skip unused bytes
			var cellHeight = reader.ReadBytes(4);
			var type = reader.ReadByte();
			reader.ReadBytes(3); // Skip unknown bytes
			var cellType = new CellType
			{
				Type = type,
				MapData = $"{Path.GetFileName(filePath),-15} [x={x + 1} - y={y + 1}]",
				Count = 1,
				IsUnderWater = (((IStructuralComparable) cellHeight).CompareTo(waterLevel, Comparer<byte>.Default) > 0)
			};
			
			bool Match(CellType o) => o.Type == cellType.Type && o.IsUnderWater == cellType.IsUnderWater;
			if (!_types.Exists(Match))
				_types.Add(cellType);
			else
				_types[_types.FindIndex(Match)].Count++;
		}
	}
}

private static byte[] GetWaterLevel(string filePath)
{
	if (!File.Exists(filePath)) return null;
	using var reader = new BinaryReader(File.OpenRead(filePath));
	reader.ReadBytes(4); // Skip magic header
	var majorVersion = reader.ReadByte();
	var minorVersion = reader.ReadByte();
	reader.ReadBytes((majorVersion > 1 || (majorVersion == 1 && minorVersion >= 4)) ? 160 : 120); // Skip unused bytes
	return reader.ReadBytes(4);
}

 

 

Any thoughts/suggestions?

 

 

~Kenpachi

Edited November 18, 2019 by Kenpachi
fixed typo / text highlighting

[Asheraf 123]

Oops, my mistake i mixed up the checks... it does indeed check for 1 element (a 4 bytes float).

This is the struct for cell attributes in aegis, and the check against water level uses h1

struct AttrCell
{
	float h1;
	float h2;
	float h3;
	float h4;
	int flag;
};

*Edit*: corrected the code in previous post

[Kenpachi 65]

Thanks a lot. That's something to work with. 😊

But this actually means, that *Athenas interpretation of water cells is far from official behavior.
According to your code snippet every cell can be a water cell regardless of its type, but *Athena only accept type 3 or alternatively type 0 (cast to 3) if the conditions are fulfilled:

// FROM map.c - map_readgat(struct map_data *m)

if( type == 0 && water_height != NO_WATER && height > water_height )
	type = 3; // Cell is 0 (walkable) but under water level, set to 3 (walkable water)

 

And additionally I'm wondering about the difference in the water level check condition. According to your code snippet AEGIS checks if cell height is less than water level, while everyone else checks if cell height is greater than water level. This really confuses me. 😲


~Kenpachi

Edited November 20, 2019 by Kenpachi

[Asheraf 123]

Again typo on my side, switched the place of variables and forgot to switch the sign 😂

[Kenpachi 65]

No Problem, dude. You really helped me. 😍
I think my question was answered completely now.

Maybe you should consider to discuss Hercules' way of interppreting water cells with the dev team. (Refering to my last post.)


*Marking your answer as solution.*


~Kenpachi

Edited November 21, 2019 by Kenpachi

[Naruto 38]

so what about the eight path algorythm that is so different from rathena and herc? 

Edited December 27, 2019 by Naruto