New core/version 0.3

News from administrators.
beannaich
Posts: 149
Joined: Mon Oct 21, 2013 2:43 pm

Re: New core/version 0.3

Post by beannaich » Thu Jul 31, 2014 8:10 pm

MarathonMan wrote:I'll still likely keep MSVC builds around somehow, as beannaich has been helping to setup some CI system w/ build agents.
I am testing the bamboo builds tonight, if that doesn't work as well as planned, TeamCity will definitely be able to handle it, I'll just have to go through the motions of setting that up next week. After that, I'll be putting the finishing touches on my compatibility database and looking to roll that out as well.

I've been thinking of ways to do automated regression testing, I have a few ideas :-) perhaps a distributed computing system with multiple "game player agents" running that make auto-generated compatibility database reports would be the best way of maintaining an up-to-date compatibility listing? It's certainly not above my pay-grade ;) All that would have to be worked out is some sort of metric system to determine if a given ROM gets to certain point, etc.

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Thu Jul 31, 2014 8:48 pm

Bump. I pushed a batch of commits. They slightly improvement performance slightly and drastically reduce the binary size. I'm getting a ~48KiB binary on my system, ~44KiB of which is actual machine code).
Narann wrote:I just bounce on the "it could be great to support multiple compilers".

Maybe it's not time for that yet but when the time to support binary bundles to end user, I suggest you choose a compiler to support build for (bugfix, performance, etc...) and, if you want (debugging is a valid point), add compiling/project support for other compilers on the dev side.
Hmm, I agree that it would be confusing to have too many binaries. It's most likely that as krom said, use GCC for all "official" builds (but provide Clang, MSVC, etc. on the side).
Narann wrote:2) Keep a consistent code and avoid a lot of #pragma or "compiler abstraction layer" headers* depending on compilers.
The common.h header contains just about all the compiler-specific wizardry that I hope to ever put in the project. There are two exceptions (though the first one is a big one!):
  • RSP, RDP, VR4300 CP1, etc. backends -- these will use intrinsics (i.e,. SSE) and inline assembly.
  • Compiler-specific machinery for doing 128-bit multiplies and divides.
Narann wrote:compiler abstraction layer" headers is an inefficient practice on a performance side (IMHO) because it considere every compiler work the same way and only some keywords changes.
I'm not one to often say this, but sometimes you have to sacrifice performance for design. It's not something I often do, but it has to be! If this project is going to support a wide-variety of operating systems and compilers, it's just too much book-keeping to not have abstraction. If I didn't have some kind of compiler/architecture abstraction layers, it'd be a nightmare to maintain accuracy across all platforms, even with the world's best CI tools.
Narann wrote:Maybe you already think about this but the choice for the "main compiler" (the one you will focus performance sensitive work) is not something trivial. To be honnest, I would also push to choose one version of gcc to focus on an maybe update this version every year.
I'm highly opposed to compiler-specific (and even worse IMO, version-specific) hacks. Although you can squeeze out a couple extra VI/s in some cases, those same "improvements" can later become regressions due to simple various code additions and bug fixes, even when compiled with the same compiler!

Ironically, CEN64 performance has tended to drift upwards with every release of GCC. I started development on 4.6 or 4.7, and have seen improvements through 4.8 and 4.9.
beannaich wrote:I've been thinking of ways to do automated regression testing, I have a few ideas :-) perhaps a distributed computing system with multiple "game player agents" running that make auto-generated compatibility database reports would be the best way of maintaining an up-to-date compatibility listing? It's certainly not above my pay-grade ;) All that would have to be worked out is some sort of metric system to determine if a given ROM gets to certain point, etc.
Typically, how I've done it with my bigger-class architectural simulators is to find a hole in the opcode space and implement an instruction that is undefined on the architecture (or toss something in the memory-mapped I/O space, whatever else -- purists are probably clenching their fists right now :P). When the simulation encounters this instruction, simulation stops and control flows back to the host who them inspects the state of the simulation for any regressions in accuracy.

This apparatus could be extended with a timer that tracks the amount of time spent on each unit test and seeks out any performance regressions as well.

beannaich
Posts: 149
Joined: Mon Oct 21, 2013 2:43 pm

Re: New core/version 0.3

Post by beannaich » Thu Jul 31, 2014 9:37 pm

MarathonMan wrote:Typically, how I've done it with my bigger-class architectural simulators is to find a hole in the opcode space and implement an instruction that is undefined on the architecture (or toss something in the memory-mapped I/O space, whatever else -- purists are probably clenching their fists right now :P). When the simulation encounters this instruction, simulation stops and control flows back to the host who them inspects the state of the simulation for any regressions in accuracy.

This apparatus could be extended with a timer that tracks the amount of time spent on each unit test and seeks out any performance regressions as well.
Sounds workable :) I like the idea of a distributed "QA" type of system, one could run on a Linux host, one on a Windows host, to weed out differences in OSes and even different hardware setups.

Is there anything special required to build CEN64? OpenGL, OpenAL, etc? I'm going to test out the bamboo build now, and I'd like to save as much time as possible.

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Thu Jul 31, 2014 10:06 pm

beannaich wrote:Sounds workable :) I like the idea of a distributed "QA" type of system, one could run on a Linux host, one on a Windows host, to weed out differences in OSes and even different hardware setups.

Is there anything special required to build CEN64? OpenGL, OpenAL, etc? I'm going to test out the bamboo build now, and I'd like to save as much time as possible.
For now, OpenGL and a C library are the only hard requirements across all platforms. Windows requires Winsock2 in addition to those requirements. And, yes, OpenAL will be required in the future (though does not currently get linked in).

krom
Posts: 72
Joined: Sat Oct 05, 2013 2:19 am

Re: New core/version 0.3

Post by krom » Thu Jul 31, 2014 10:11 pm

Hi MarathonMan, the new cen64 is running faster than ever with your latest optimizations well done =D
I have noticed however that on windows -O3 is much faster than the -O2 optimization flag, maybe the changes you have made, makes it perform better now with -O3 in Linux?
I get about a 10% speed increase across all demos using -O3, it was most noticable in "Christmas Flame Demo by Halley's Comet Software (PD).z64" where I could see the snow falling much quicker!
Would be interesting if Linux shows the same speedup, or it could be a platform specific thing... =D

P.S I just noticed I am using -funsafe-loop-optimizations & -finline-limit=512 in my makefile compile options, which are the only differences to your CMakeLists.txt compile options.
They could be another reason for the speedup...

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Thu Jul 31, 2014 10:42 pm

krom wrote:I have noticed however that on windows -O3 is much faster than the -O2 optimization flag, maybe the changes you have made, makes it perform better now with -O3 in Linux?

...

P.S I just noticed I am using -funsafe-loop-optimizations & -finline-limit=512 in my makefile compile options, which are the only differences to your CMakeLists.txt compile options.
They could be another reason for the speedup...
I tried -O3/-funsafe-loop-optimizations -finline-limit=512, but my performance absolutely tanks whenever I use those -- to the tune of 10-20%+ across a whole bunch of ROMs.

This is more likely due to a difference in microarchitecture than it is OS (as the compilers are the same, and CEN64 has a limited number of kernel/system/library calls).

What CPU do you have?

krom
Posts: 72
Joined: Sat Oct 05, 2013 2:19 am

Re: New core/version 0.3

Post by krom » Thu Jul 31, 2014 10:50 pm

MarathonMan wrote:What CPU do you have?
I have an old 1st gen I7 940 2.9 GHz =D

No probs, I just thought I would mention it in case it could help you get more VI's!

beannaich
Posts: 149
Joined: Mon Oct 21, 2013 2:43 pm

Re: New core/version 0.3

Post by beannaich » Thu Jul 31, 2014 11:22 pm

Code: Select all

vi\controller.c(58): warning C4244: '=' : conversion from 'float' to 'unsigned int', possible loss of data [C:\Users\Adam\bamboo-home\xml-data\build-dir\CEN64-REL-JOB1\cen64.vcxproj]
vi\controller.c(80): error C2065: 'GL_UNSIGNED_SHORT_5_5_5_1' : undeclared identifier [C:\Users\Adam\bamboo-home\xml-data\build-dir\CEN64-REL-JOB1\cen64.vcxproj]
(ClCompile target) -> 
  vi\controller.c(80): error C2065: 'GL_UNSIGNED_SHORT_5_5_5_1' : undeclared identifier [C:\Users\Adam\bamboo-home\xml-data\build-dir\CEN64-REL-JOB1\cen64.vcxproj]
What version of glfw.h do I need for this to compile? All else is working just fine.

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Thu Jul 31, 2014 11:36 pm

beannaich wrote:

Code: Select all

vi\controller.c(58): warning C4244: '=' : conversion from 'float' to 'unsigned int', possible loss of data [C:\Users\Adam\bamboo-home\xml-data\build-dir\CEN64-REL-JOB1\cen64.vcxproj]
vi\controller.c(80): error C2065: 'GL_UNSIGNED_SHORT_5_5_5_1' : undeclared identifier [C:\Users\Adam\bamboo-home\xml-data\build-dir\CEN64-REL-JOB1\cen64.vcxproj]
(ClCompile target) -> 
  vi\controller.c(80): error C2065: 'GL_UNSIGNED_SHORT_5_5_5_1' : undeclared identifier [C:\Users\Adam\bamboo-home\xml-data\build-dir\CEN64-REL-JOB1\cen64.vcxproj]
What version of glfw.h do I need for this to compile? All else is working just fine.
I dumped GLFW for pure WinAPI. You must not be passing the right arguments to the compiler or something...?

Code: Select all

./os/gl_window.h:#define GL_UNSIGNED_SHORT_5_5_5_1 0x8034
krom wrote:
MarathonMan wrote:What CPU do you have?
I have an old 1st gen I7 940 2.9 GHz =D

No probs, I just thought I would mention it in case it could help you get more VI's!
That explains things. The uop cache that I've been trying to optimize around is only present in SNB (i*-2xxx cores), IVB (i*-3xxx) and Haswell (i*-4xxx). If you don't have any of these, you probably want a binary that's a little more unrolled.

beannaich
Posts: 149
Joined: Mon Oct 21, 2013 2:43 pm

Re: New core/version 0.3

Post by beannaich » Thu Jul 31, 2014 11:40 pm

MarathonMan wrote:I dumped GLFW for pure WinAPI. You must not be passing the right arguments to the compiler or something...?

Code: Select all

./os/gl_window.h:#define GL_UNSIGNED_SHORT_5_5_5_1 0x8034
unfortunately, I don't have the luxury of editing files before a build. The entire build is automated and I don't feel like writing a script to hack around that constant. :(

The current script just runs CMake, and then MSBuild with the generated *.sln file.. Not adding any other parameters (although, I can add parameters to the MSBuild step if that would help).

Code: Select all

# Source Code Checkout [Build Step]
# Script [Build Step]

cmake -G "Visual Studio 12 2013"

rmdir /s /q GL
mkdir GL

copy "%C_PATH%include\GL\glfw.h" "${bamboo.build.working.directory}\GL\glfw.h"

# MSBuild [Build Step]

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Thu Jul 31, 2014 11:47 pm

beannaich wrote:
MarathonMan wrote:I dumped GLFW for pure WinAPI. You must not be passing the right arguments to the compiler or something...?

Code: Select all

./os/gl_window.h:#define GL_UNSIGNED_SHORT_5_5_5_1 0x8034
unfortunately, I don't have the luxury of editing files before a build. The entire build is automated and I don't feel like writing a script to hack around that constant. :(

The current script just runs CMake, and then MSBuild with the generated *.sln file.. Not adding any other parameters (although, I can add parameters to the MSBuild step if that would help).

Code: Select all

# Source Code Checkout [Build Step]
# Script [Build Step]

cmake -G "Visual Studio 12 2013"

rmdir /s /q GL
mkdir GL

copy "%C_PATH%include\GL\glfw.h" "${bamboo.build.working.directory}\GL\glfw.h"

# MSBuild [Build Step]
I'm saying it should clean build. I'll do a CMake checkout and VS2013 build now... give me a minute.

EDIT: Just checked out master into a clean folder, fired up CMake GUI. Clicked configure, generate. Double clicked VS2013 solution file, selected release build. Built first time.

:?:

beannaich
Posts: 149
Joined: Mon Oct 21, 2013 2:43 pm

Re: New core/version 0.3

Post by beannaich » Thu Jul 31, 2014 11:54 pm

MarathonMan wrote:Just checked out master into a clean folder, fired up CMake GUI. Clicked configure, generate. Double clicked VS2013 solution file, selected release build. Built first time.

:?:
Maybe the GUI is adding something to the generator step that I'm missing when running it in the command line? Running the solution in VS2013 manually I get the same "GL not found error" I was getting before I started copying it from my C++/include folder.

EDIT: I'd like to add, I don't see an "os" folder anywhere..
EDIT2: it seems if I use the git:// instead of http:// i get the "os" folder.
Last edited by beannaich on Fri Aug 01, 2014 12:01 am, edited 1 time in total.

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Fri Aug 01, 2014 12:00 am

beannaich wrote:EDIT: I'd like to add, I don't see an "os" folder anywhere..
You've got some serious git issues going on your end if you don't see an 'os' folder.

Are you checking out from the new git server, or GitHub? GitHub only has the old core.
git://git.cen64.com/cen64.git
http://git.cen64.com/cen64.git

Current snapshot in .tar.gz format:
http://git.cen64.com/?p=cen64.git;a=sna ... ter;sf=tgz

beannaich
Posts: 149
Joined: Mon Oct 21, 2013 2:43 pm

Re: New core/version 0.3

Post by beannaich » Fri Aug 01, 2014 12:02 am

MarathonMan wrote:
beannaich wrote:EDIT: I'd like to add, I don't see an "os" folder anywhere..
You've got some serious git issues going on your end if you don't see an 'os' folder.

Are you checking out from the new git server, or GitHub? GitHub only has the old core.
git://git.cen64.com/cen64.git
http://git.cen64.com/cen64.git

Current snapshot in .tar.gz format:
http://git.cen64.com/?p=cen64.git;a=sna ... ter;sf=tgz
I am of course checking out from the new repository. I was checking out from http:// but I'll try git:// as that seems to be giving me the correct sources.

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Fri Aug 01, 2014 12:09 am

beannaich wrote:I am of course checking out from the new repository. I was checking out from http:// but I'll try git:// as that seems to be giving me the correct sources.
LOL, not sure what strange planet the http one points to. Going to fix that; sorry for the confusion.

beannaich
Posts: 149
Joined: Mon Oct 21, 2013 2:43 pm

Re: New core/version 0.3

Post by beannaich » Fri Aug 01, 2014 12:16 am

MarathonMan wrote:LOL, not sure what strange planet the http one points to. Going to fix that; sorry for the confusion.
Not a problem, new error:

Code: Select all

Error	11	error MSB3721: The command "ml.exe /c /nologo /Zi /Fo"cen64.dir\Release\fpu_cmp_eq_32.obj" /W3 /errorReport:prompt  /Taos\windows\fpu\x86_64\fpu_cmp_eq_32.asm" exited with code 1.	C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120\BuildCustomizations\masm.targets	50	5	cen64
This is happening in the build server, and in VS2013 with a fresh clone of git:// and "cmake -G "Visual Studio 12 2013"

EDIT: The error message looks pretty broken, I wonder if my install is borked.
EDIT2: Running with ml64.exe works better, is there a switch for generating a 64-bit build in CMake?

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Fri Aug 01, 2014 12:23 am

Just to sort out any possible git problems, here's my copy of the file:

Code: Select all

.code
fpu_cmp_eq_32 proc
  movss xmm0, DWORD PTR [rcx]
  movss xmm1, DWORD PTR [rdx]
  comiss xmm1, xmm0
  sete dl
  setnp al
  and al, dl
  ret 
fpu_cmp_eq_32 endp
end

beannaich
Posts: 149
Joined: Mon Oct 21, 2013 2:43 pm

Re: New core/version 0.3

Post by beannaich » Fri Aug 01, 2014 12:29 am

Won't be needed:

Image

:D

To anyone wondering, the proper CMake command is:

Code: Select all

cmake -G "Visual Studio 12 Win64"

User avatar
Snowstorm64
Posts: 303
Joined: Sun Oct 20, 2013 8:22 pm

Re: New core/version 0.3

Post by Snowstorm64 » Fri Aug 01, 2014 8:55 am

Whoa! With CEN64's newest optimizations, Plasma Demo just gained another 9 VI/s, running at 94 VI/s! And for all other demos it just gained, generally, 5 VI/s! :D

EDIT: @MarathonMan: I have reported this a few days ago, but maybe you have missed it so I'm reposting this. If you try to boot My Angel Demo (PD) there is the output:

Code: Select all

Unimplemented instruction: SYNC [0x0000000F] @ 0xFFFFFFFF80310234
cen64-debug: /data/emulators/cen64/vr4300/functions.c:779: VR4300_INV: Assertion `0 && "Unimplemented instruction encountered."' failed.
OS: Debian GNU/Linux Jessie (8.0)
CPU: Intel i7 4770K @ 3.5 GHz
Build: AVX (compiled from git)

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Fri Aug 01, 2014 9:59 am

Snowstorm64 wrote:EDIT: @MarathonMan: I have reported this a few days ago, but maybe you have missed it so I'm reposting this. If you try to boot My Angel Demo (PD) there is the output:

Code: Select all

Unimplemented instruction: SYNC [0x0000000F] @ 0xFFFFFFFF80310234
cen64-debug: /data/emulators/cen64/vr4300/functions.c:779: VR4300_INV: Assertion `0 && "Unimplemented instruction encountered."' failed.
Oops, didn't see that. Regardless, from the manual:
The SYNC instruction is executed as a NOP on the VR4300. This operation
maintains compatibility with code that conforms to the VR4400.
This instruction is defined to maintain software compatibility with the VR4400.
So it can safely be ignored. I'll patch it later.

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Fri Aug 01, 2014 1:06 pm

Took like 15 minutes and shoehorned the RDP from the old CEN64 core into the new one. Enjoy:

http://git.cen64.com/?p=cen64-backport.git;a=summary
  • You'll have to build with NATIVE_BUILD set to ON.
  • It'll probably also only build on GCC (or Clang).
  • I won't fix or look at any bugs in this repo.
Hopefully, I'll get another short break tonight and do the same for the RSP. I need to be able to run more code through the new core than just simple RD ROMs, so let the games begin.

User avatar
Snowstorm64
Posts: 303
Joined: Sun Oct 20, 2013 8:22 pm

Re: New core/version 0.3

Post by Snowstorm64 » Fri Aug 01, 2014 5:33 pm

Isn't better to implement SDL/LDL and TLB instructions before merging RSP and RDP modules into CEN64 (though the git one with -backports)? I mean...a lot of commercial ROMs want at least one of these instructions to work properly.
OS: Debian GNU/Linux Jessie (8.0)
CPU: Intel i7 4770K @ 3.5 GHz
Build: AVX (compiled from git)

krom
Posts: 72
Joined: Sat Oct 05, 2013 2:19 am

Re: New core/version 0.3

Post by krom » Fri Aug 01, 2014 5:35 pm

MarathonMan wrote:Took like 15 minutes and shoehorned the RDP from the old CEN64 core into the new one. Enjoy
Great Work MarathonMan, in only 15 mins is amazing!!
MarathonMan wrote:Hopefully, I'll get another short break tonight and do the same for the RSP.
Will be amazing if you can get this done tonight too, very exciting moment for cen64 =D

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Fri Aug 01, 2014 6:41 pm

Snowstorm64 wrote:Isn't better to implement SDL/LDL and TLB instructions before merging RSP and RDP modules into CEN64 (though the git one with -backports)? I mean...a lot of commercial ROMs want at least one of these instructions to work properly.
LDL/R already is implemented. ;)

SDL/R is probably pretty trivial, too.

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Sat Aug 02, 2014 1:03 pm

RSP is in.

Some serious performance regressions going on; not sure what they're attributable to? Some ROMs don't boot probably because the PI or something isn't well developed enough.

On the plus side, the few bugs I checked for
* SPLiT's Nacho (PD) - Eventual crash.
* Mario Kart 64 (U) - Corrupt graphics when built with LLVM

Are now gone. :)

User avatar
Snowstorm64
Posts: 303
Joined: Sun Oct 20, 2013 8:22 pm

Re: New core/version 0.3

Post by Snowstorm64 » Sat Aug 02, 2014 2:11 pm

I wonder if it's RDP or RSP fault...after the core being rewritten entirely and it having passed all krom's tests, not much has changed. Maybe will worth it to rewrite RDP/RSP? Maybe rebasing the code on Iconoclast's plugins...
OS: Debian GNU/Linux Jessie (8.0)
CPU: Intel i7 4770K @ 3.5 GHz
Build: AVX (compiled from git)

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Sat Aug 02, 2014 3:30 pm

Snowstorm64 wrote:I wonder if it's RDP or RSP fault...after the core being rewritten entirely and it having passed all krom's tests, not much has changed. Maybe will worth it to rewrite RDP/RSP? Maybe rebasing the code on Iconoclast's plugins...
Rewriting the RDP/RSP from scratch has been the goal all along. I just want to backport the old versions so I can focus on fixing bugs in one component at a time.

No interest in rebasing off Iconoclast's plugins. He won't use the SSE intrinsics and isn't cycle-accurate anyways. Most of the time consuming portion of RSP development is all the vector operations and load/stores.

User avatar
Snowstorm64
Posts: 303
Joined: Sun Oct 20, 2013 8:22 pm

Re: New core/version 0.3

Post by Snowstorm64 » Sat Aug 02, 2014 3:49 pm

Ok, understood. I guess it will need quite some time for new RDP/RSP to shape. And I will be here to test them. :)
OS: Debian GNU/Linux Jessie (8.0)
CPU: Intel i7 4770K @ 3.5 GHz
Build: AVX (compiled from git)

User avatar
mikewando
Posts: 1
Joined: Sat Aug 02, 2014 3:57 pm

Re: New core/version 0.3

Post by mikewando » Sat Aug 02, 2014 4:57 pm

So figured I'd try out cen64-backport today, but I've run into several problems building on windows that I figured I'd detail here for others.

First I tried compiling with Visual Studio 2012 but that choked on "uint8_t byteval, c;" and http://stackoverflow.com/questions/9903 ... expression seemed a likely explanation, so I installed Visual Studio 2013 express and tried again. But then I got the same error, just later. Now it errored on

Code: Select all

  if (other_modes.cycle_type == CYCLE_TYPE_FILL || other_modes.cycle_type == CYCLE_TYPE_COPY)
    yl |= 3;

  uint32_t xlint = (xl >> 2) & 0x3ff;
Digging through the comments on the stackoverflow issue I saw
So turns out it really was a bug, and apparently it only happens when the new variable comes after a block statement without braces. Workaround: add explicit {} braces.
Indeed adding braces fixed the issue, and then I had to change several more instances to have braces.

I also had to include <windows.h> if WIN32 was defined in core.c since it was complaining about MB_OK being undefined.

Finally I ran into an unresolved symbol __builtin_bswap32. Most of this was fixed by changing the bswap32 functions in Core.c and FBAccess.c to return a non-builtin version if WIN32 is defined (probably better to only return the builtin if we're sure the compiler has it, but I don't know what that ifdef would look like), but there was once instance in rdp_process_list which just used the straight __builtin_bswap32 without an intermediate and so I changed that to bswap32.

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Sat Aug 02, 2014 5:55 pm

Snowstorm64 wrote:I wonder if it's RDP or RSP fault...after the core being rewritten entirely and it having passed all krom's tests, not much has changed. Maybe will worth it to rewrite RDP/RSP? Maybe rebasing the code on Iconoclast's plugins...
Ended up being neither. :lol: I saw that the PC kept spinning around 0x800004xx or something. Then I remembered: Oh yeah, I only hacked the security check to pass 6102 CICs only. Replacing the PIF response bytes for 6105 carts booted all 6105 ROMs. :D

EDIT: Regressions fixed too! Whoopie.

User avatar
Snowstorm64
Posts: 303
Joined: Sun Oct 20, 2013 8:22 pm

Re: New core/version 0.3

Post by Snowstorm64 » Sat Aug 02, 2014 7:35 pm

MarathonMan, you better rewrite RDP/RSP or.... :P

That regression affected Mario Party 1&2. Now they boot.

Regarding Mario Party, I found another bug (unrelated to cen64-backport, just the regular one):

Code: Select all

cen64-debug: /data/emulators/cen64/si/controller.c:71: pif_perform_command: Assertion `0 && "Invalid channel."' failed.
What is this?

Also, in debug mode, it still complaining about unimplemented DCACHE operations, I thought you had implemented it?
OS: Debian GNU/Linux Jessie (8.0)
CPU: Intel i7 4770K @ 3.5 GHz
Build: AVX (compiled from git)

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Sat Aug 02, 2014 10:51 pm

A large number of the ROMs that worked prior to the old core (excepting TLB) are working on my end (once you set the CIC seed).

The PIF channel dealio means that the PIF tried to talk on something other than channel 1-4, which is bad news bears.

EDIT: Only implemented ICACHE.

krom
Posts: 72
Joined: Sat Oct 05, 2013 2:19 am

Re: New core/version 0.3

Post by krom » Sun Aug 03, 2014 11:31 am

Hi MarathonMan,
Great work on the new backport cen64, which is really impressive with the RSP & RDP inside!
Loads of stuff now boots that did not in the old version of cen64, like Blastcorps =D
I am gonna now convert my LLE RDP GPU driver to the new style, to test out how much of a speed improvement a minimal optimized RDP core would make to the framerate on my system...

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Sun Aug 03, 2014 11:44 am

krom wrote:Hi MarathonMan,
Great work on the new backport cen64, which is really impressive with the RSP & RDP inside!
Loads of stuff now boots that did not in the old version of cen64, like Blastcorps =D
I am gonna now convert my LLE RDP GPU driver to the new style, to test out how much of a speed improvement a minimal optimized RDP core would make to the framerate on my system...
Great! If you want to get push access to either cen64-backports (or your own git repo), just PM me and we'll get things set up. Or not... your choice! :P

User avatar
Snowstorm64
Posts: 303
Joined: Sun Oct 20, 2013 8:22 pm

Re: New core/version 0.3

Post by Snowstorm64 » Sun Aug 03, 2014 12:06 pm

To tell the truth, Blast Corps did boot indeed with old core, it just stopped at main screen. And with new core it still suffers the same issue it had on old cen64. Anyways...regarding to same game, I found another unimplemented instruction:

Code: Select all

Unimplemented instruction: DMTC1 [0x44B40000] @ 0xFFFFFFFF802A4308
cen64-debug: /data/emulators/cen64-backport/vr4300/functions.c:780: VR4300_INV: Assertion `0 && "Unimplemented instruction encountered."' failed.
Hey you, Pikachu! now boots, but with debug mode it outputs this:

Code: Select all

cen64-debug: /data/emulators/cen64-backport/vr4300/cp1.c:1260: VR4300_MFC1: Assertion `!((((iw) >> 11 & 0x1F) + VR4300_REGISTER_CP1_0) & 0x1)' failed.
EDIT: I could sworn that Hey you, Pikachu did boot before, now doesn't do that anymore.

Killer Instinct Gold

Code: Select all

Unimplemented ICACHE operation: 0
What about now, MarathonMan? :P
Last edited by Snowstorm64 on Sun Aug 03, 2014 12:26 pm, edited 1 time in total.
OS: Debian GNU/Linux Jessie (8.0)
CPU: Intel i7 4770K @ 3.5 GHz
Build: AVX (compiled from git)

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Sun Aug 03, 2014 12:26 pm

Snowstorm64 wrote:To tell the truth, Blast Corps did boot indeed with old core, it just stopped at main screen. And with new core it still suffers the same issue it had on old cen64. Anyways...regarding to same game, I found another unimplemented instruction:

Code: Select all

Unimplemented instruction: DMTC1 [0x44B40000] @ 0xFFFFFFFF802A4308
cen64-debug: /data/emulators/cen64-backport/vr4300/functions.c:780: VR4300_INV: Assertion `0 && "Unimplemented instruction encountered."' failed.
DK64 uses those instructions too. They're very common.
Snowstorm64 wrote:Hey you, Pikachu! now boots, but with debug mode it outputs this:

Code: Select all

cen64-debug: /data/emulators/cen64-backport/vr4300/cp1.c:1260: VR4300_MFC1: Assertion `!((((iw) >> 11 & 0x1F) + VR4300_REGISTER_CP1_0) & 0x1)' failed.
That reminder I left for myself is becoming more and more of a sore thumb. :lol: I'm still undecided as to how to fix that one...
Snowstorm64 wrote:EDIT: I could sworn that Hey you, Pikachu did boot before, now doesn't do that anymore.
Probably did. The new core is still lacking a lot of things that even the old one had.

User avatar
Snowstorm64
Posts: 303
Joined: Sun Oct 20, 2013 8:22 pm

Re: New core/version 0.3

Post by Snowstorm64 » Sun Aug 03, 2014 12:34 pm

Oh, I understand it.


Reposting this, in case you did miss it:
Snowstorm64 wrote: Killer Instinct Gold

Code: Select all

Unimplemented ICACHE operation: 0
What about now, MarathonMan? :P
Report: Mario Golf now boots on cen64 for first time ever!
Report: Mario Party 3 prints this:

Code: Select all

cen64-debug: /data/emulators/cen64-backport/pi/controller.c:130: write_cart_rom: Assertion `0 && "Attempt to write to cart ROM."' failed.
Report: This one is about the RSP
Mortal Kombat 64

Code: Select all

cen64-debug: /data/emulators/cen64-backport/rsp/Memory.c:350: LoadPackedVector: Assertion `0' failed.
Report: Quake II

Code: Select all

cen64-debug: /data/emulators/cen64-backport/vr4300/functions.c:368: VR4300_CACHE: Assertion `!segment->mapped' failed.
Report: My Angel Demo

Code: Select all

cen64-debug: /data/emulators/cen64-backport/pi/controller.c:79: pi_dma_write: Assertion `0' failed.
Last edited by Snowstorm64 on Sun Aug 03, 2014 12:54 pm, edited 1 time in total.
OS: Debian GNU/Linux Jessie (8.0)
CPU: Intel i7 4770K @ 3.5 GHz
Build: AVX (compiled from git)

krom
Posts: 72
Joined: Sat Oct 05, 2013 2:19 am

Re: New core/version 0.3

Post by krom » Sun Aug 03, 2014 12:43 pm

Just tried out my LLE GL Driver & everything is performing very fast, some stuff is 30-40% faster!!
MarathonMan wrote:Great! If you want to get push access to either cen64-backports (or your own git repo), just PM me and we'll get things set up. Or not... your choice!
Heh, there are a few things I would like todo before releasing openly:
1. I am working on some rdp textured triangle tests at the mo, I will then be able to set up the OpenGL textured triangles correctly.
2. Textured rectangles are already in and pass every single one of my RDP/TextureRectangle tests, however my textured rectangles need a few fixups to be perfect like HW.
3. I need todo some ZBuffer triangle RDP tests to make ZBuffered triangles perfect too.

I'll give you P.Ms with my RDP OpenGL driver updates, if you feel it is good enough to have it's own repo, I'll be happy to release & update the repo etc. =D

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Sun Aug 03, 2014 2:09 pm

Snowstorm64 wrote:Report: Mario Golf now boots on cen64 for first time ever!
So it was true... those bugs that I found when I was rewriting, however minor, made a difference. Great! :P
Snowstorm64 wrote:Report: Mario Party 3 prints this:

Code: Select all

cen64-debug: /data/emulators/cen64-backport/pi/controller.c:130: write_cart_rom: Assertion `0 && "Attempt to write to cart ROM."' failed.
Almost certainly a missing assert(segment->mapped) before bus_write_word in the VR4300.

tl;dr: Needs TLB.
Snowstorm64 wrote:Report: This one is about the RSP
Mortal Kombat 64

Code: Select all

cen64-debug: /data/emulators/cen64-backport/rsp/Memory.c:350: LoadPackedVector: Assertion `0' failed.
These functions were hastily done and almost certainly have errors/unimplemented cases... probably the same reason that the bars appear in OoT intro when the camera sits over on top of Link's room.
Snowstorm64 wrote:Report: Quake II

Code: Select all

cen64-debug: /data/emulators/cen64-backport/vr4300/functions.c:368: VR4300_CACHE: Assertion `!segment->mapped' failed.
Needs TLB.
Snowstorm64 wrote:Report: My Angel Demo

Code: Select all

cen64-debug: /data/emulators/cen64-backport/pi/controller.c:79: pi_dma_write: Assertion `0' failed.
Basically means it tried to read past the cart's size... not sure what to do in this case. Will have to check with MAME/hardware to see what happens.

beannaich
Posts: 149
Joined: Mon Oct 21, 2013 2:43 pm

Re: New core/version 0.3

Post by beannaich » Sun Aug 03, 2014 2:57 pm

MarathonMan wrote:Basically means it tried to read past the cart's size... not sure what to do in this case. Will have to check with MAME/hardware to see what happens.
Most likely caused by multi-chip cartridges, giving an odd number of megabytes (for example, 1x1 MiB + 1x2 MiB = 3 MiB). What almost always happens is that the cartridge space is "padded" by repeating copies of the smaller chips.

For example, consider a fake system with 8 KiB of total cartridge space. The cartridge takes up 3 KiB due to a multi-chip configuration. Reading the first 2 KiB works as expected ($0000-$07ff), as does the 1 KiB after that ($0800-$0bff) but the next 1 KiB ($0c00-$0fff) would return the same data as $0800-$0bff. The pattern would repeat with the first 2 KiB mapped into $1000 unless a full address decoding was done (not usually the case in most systems I've seen).

This happens because of how the ROM chips are coded into the address space. This happens on GBA and SNES, so I would assume it's the same for N64.

User avatar
Snowstorm64
Posts: 303
Joined: Sun Oct 20, 2013 8:22 pm

Re: New core/version 0.3

Post by Snowstorm64 » Sun Aug 03, 2014 3:29 pm

I just discovered an interesting fact: F-Zero X, Yoshi's Story and Cruis'n World all share same CIC Chip (CIC-NUS-6106). If you try to debug them, all three output the same error:

Code: Select all

Unimplemented instruction: INVALID [0x7BE736AE] @ 0xFFFFFFFF80000004
cen64-debug: /data/emulators/cen64-backport/vr4300/functions.c:780: VR4300_INV: Assertion `0 && "Unimplemented instruction encountered."' failed.
So to fix it, we need to implement the PIF response for 6106.

I guess a mystery have just been resolved. :)
OS: Debian GNU/Linux Jessie (8.0)
CPU: Intel i7 4770K @ 3.5 GHz
Build: AVX (compiled from git)

krom
Posts: 72
Joined: Sat Oct 05, 2013 2:19 am

Re: New core/version 0.3

Post by krom » Sun Aug 03, 2014 4:01 pm

Hi MarathonMan,
Your latest commit broke the LWR instruction (Check my CPUTest/CPU/LOADSTORE/CPULW.N64 rom), which in turn has broken Fushigi no Dungeon - Fuurai no Shiren 2 - Oni Shuurai! Shiren Jou! (J) intro which was working perfectly before...
Hope this helps =D

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Sun Aug 03, 2014 5:05 pm

krom wrote:Hi MarathonMan,
Your latest commit broke the LWR instruction (Check my CPUTest/CPU/LOADSTORE/CPULW.N64 rom), which in turn has broken Fushigi no Dungeon - Fuurai no Shiren 2 - Oni Shuurai! Shiren Jou! (J) intro which was working perfectly before...
Hope this helps =D
I would have lost my head looking for that typo... thanks!

krom
Posts: 72
Joined: Sat Oct 05, 2013 2:19 am

Re: New core/version 0.3

Post by krom » Sun Aug 03, 2014 5:11 pm

MarathonMan wrote:I would have lost my head looking for that typo... thanks!
No probs, it's precisely why I coded the tests... to make it easy for us to check & fix bugs =D

User avatar
Snowstorm64
Posts: 303
Joined: Sun Oct 20, 2013 8:22 pm

Re: New core/version 0.3

Post by Snowstorm64 » Sun Aug 03, 2014 5:37 pm

krom wrote:
MarathonMan wrote:I would have lost my head looking for that typo... thanks!
No probs, it's precisely why I coded the tests... to make it easy for us to check & fix bugs =D
krom, do you think a rom with all CP1/CPU tests would be feasibile? Like this. Or it would need too much work to do? It could be useful for check every commit avoiding launch every needed test one at a time.

EDIT: ooops.
Last edited by Snowstorm64 on Sun Aug 03, 2014 6:06 pm, edited 1 time in total.
OS: Debian GNU/Linux Jessie (8.0)
CPU: Intel i7 4770K @ 3.5 GHz
Build: AVX (compiled from git)

krom
Posts: 72
Joined: Sat Oct 05, 2013 2:19 am

Re: New core/version 0.3

Post by krom » Sun Aug 03, 2014 5:57 pm

Snowstorm64 wrote:krom, do you think a rom with all CP1/CPU tests would be feasible? Like this. Or it would need too much work to do?
Yes it is indeed feasible with not too much work, however I want to complete all the CPU tests before I make super tests for each component =D
I originally intended the super tests to have screens that contain "pass" for each individual component on 1 screen, if "Fail" comes up, you can check the separate test to debug etc...

Cheers for the video link, I did not know an official Nintendo N64 test rom existed, I would really love to find it!!

User avatar
Snowstorm64
Posts: 303
Joined: Sun Oct 20, 2013 8:22 pm

Re: New core/version 0.3

Post by Snowstorm64 » Sun Aug 03, 2014 6:12 pm

Oh, I understand. :)

Yes, the N64 test cartridge exists and it's expensive to obtain because it's rare. But I think you can still find it on ebay or craigslist. As far as I know, nobody has managed to dump the rom because it uses an unusual cartridge (a particular type of flash cart, I think). But let's not go off topic so we better stop here :P.
OS: Debian GNU/Linux Jessie (8.0)
CPU: Intel i7 4770K @ 3.5 GHz
Build: AVX (compiled from git)

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Sun Aug 03, 2014 6:22 pm

Snowstorm64 wrote:Oh, I understand. :)

Yes, the N64 test cartridge exists and it's expensive to obtain because it's rare. But I think you can still find it on ebay or craigslist. As far as I know, nobody has managed to dump the rom because it uses an unusual cartridge (a particular type of flash cart, I think). But let's not go off topic so we better stop here :P.
TBH, that N64 test ROM (assuming that's the one?) looked useless for the purposes of simulation. krom's ROMs are much better for finding bugs in CEN64 because they pinpoint very precise events that occur in the hardware, instead of something overly general. If you play an audio file and it doesn't work, it could be any number of instructions or devices.

Ultimately, what I'd like to do is automate krom's test ROMs along with some other ROMs that test the pipeline functionality. The way to do this would be to have the ROMs execute traps that are unique to CEN64 and tell CEN64 that it should inspect the state against a known truth. Such an apparatus would enable a queue of tests to be integrated into a git hook and occur automatically each time a commit is pushed.

User avatar
Snowstorm64
Posts: 303
Joined: Sun Oct 20, 2013 8:22 pm

Re: New core/version 0.3

Post by Snowstorm64 » Sun Aug 03, 2014 6:43 pm

It's useless for debugging purposes, true, but it could be a great benchmark that tells if an emulator behaves the same as the real HW. It would be a great 'museum exhibit' (if you know what I mean :P) to have, but I would be fine with a similiar rom that is free and public domain, too.

EDIT: Just ran Soncrap Intro by Redbox (PD), with cen64-backport the text doesn't flicker anymore, but with regular cen64 still does. That's odd... The only difference, aside from RSP/RDP, is this commit, not present in the regular git.
OS: Debian GNU/Linux Jessie (8.0)
CPU: Intel i7 4770K @ 3.5 GHz
Build: AVX (compiled from git)

User avatar
MarathonMan
Site Admin
Posts: 692
Joined: Fri Oct 04, 2013 4:49 pm

Re: New core/version 0.3

Post by MarathonMan » Sun Aug 03, 2014 9:28 pm

It's all timing differences:

cen64-backport:

Code: Select all

 18 // Currently used a fixed value...
 19 #define MEMORY_CODE_CYCLE_DELAY 50
 20 #define MEMORY_DATA_CYCLE_DELAY 0
 21 #define ICACHE_ACCESS_DELAY 50
cen64:

Code: Select all

 18 // Currently used a fixed value...
 19 #define MEMORY_CODE_CYCLE_DELAY 50
 20 #define MEMORY_DATA_CYCLE_DELAY 5
 21 #define ICACHE_ACCESS_DELAY 50
Same reason that most emulators have problems with DK64 intro and the like... eventually CEN64 will get these nailed down.

Locked

Who is online

Users browsing this forum: No registered users and 3 guests