Any progress on the direct3d render module?

Hey,

Spark looks great and I'm seriously considering using it to add particles to my current game ( found at: http://www.roboticshed.com/ ) but I'm using directx9 so I was curious to know if there was any progress made on the direct3d rendering module mentioned on the website?

If not I'm happy to look into writing my own rendering stuff but I wouldn't want to duplicate effort if there is something already started.

directx9 renderers are available on the svn.

i'm currently working on them to remove some bugs but you can download them now.

Ah thanks! Great news.

I just grabbed the code of the SVN and will investigate it tommorow.

Any specific bugs I should be aware of?

there is a memory leak and the LineRenderer works bad, else it works quite well.

i think i will commit fixes in some hours or tomorrow.

Hi, just to let you know i m currently working on the version 2 of spark. The directX renderer will be ported and released officially under the version 2. For the time being, it will remain on the svn to be debugged and optimized.

So about you consideration of using spark to your game, if you dont have very short timelines, it might be worth waiting for the release of the version 2.

Juff wrote:

Hi, just to let you know i m currently working on the version 2 of spark. The directX renderer will be ported and released officially under the version 2. For the time being, it will remain on the svn to be debugged and optimized.

So about you consideration of using spark to your game, if you dont have very short timelines, it might be worth waiting for the release of the version 2.

Ok, thats good to know. I guess the interfaces won't be changing that much between what is on the SVN now and version 2? I don't have an especially tight timeline at all but there is probably no harm in looking over the code and pondering how it will fit into the rest of my engine.

I'll have a play around with it this week but make sure I keep coming back until version 2 is released

Thanks for all the feedback.

Ok,

I had a quick look at the code today. Direct X renderer seems to be working, at least in the DX9 Basic Demo. Awesome work guys.

The main thing I wanted to look at was how easy it would be to integrate Spark into my multithreaded engine structure.

The way my engine works is that I split all 'update' code onto one thread (Physics, animation, AI) and all the rendering code onto another thread. The two threads run in parrallel and then synch up their data once they have both finished. I wanted to see if I could get Spark working reliably in this way too.

The good news is that I think its possible. At least it seems to work

I took the basic demo and made two copies of the SPK::System, SPK::Group and SPK::Model objects one for each thread. Hopefully thats enough to keep them seperate, I wasn't sure exactly what the model object was doing but it seems to be required to create the group.

On the 'update thread' copy of the SPK::System I call the update method and on the 'render thread' copy I call the render method. The render SPK::group is added to the render SPK::system and the update SPK::group is added to the update SPK::system.

The slightly tricky bit was working out what data was needed to render the group but I think its just the Pool<Particle> 'pool' member variable that is required. I then added a little method to the SPK::Group class that copies the 'pool' member from one SPK::Group to another:

Code:

void    swapRenderData(Group* otherGroup)
{
      pool = otherGroup->getParticles();
}

...and that seemed to be enough to make the demo work (though the operator= on the pool method is broken and needed some minor repairs):

Code:

   template<class T>
   Pool<T>& Pool<T>::operator=(const Pool<T>& pool)
   {
      //if (this != *pool) // There is no != operator
      //{
         nbActive = pool.nbActive; // this was previously: nbActive = pool.nbActive();
         maxTotal = 0;
         container.reserve(pool.container.capacity());
         container = pool.container;
      //}
      return *this;
   }

Can you guys see any problems with this approach to using Spark currently or in the future?

Hi, spark was not designed to be used in seperate threads atm. With your method, you actually render particles with the data of the groups of the updates thread. If it works it is simply a matter of luck because update and render dont happen concurrently.

Why is that ? This is due to the way particles data are stored internally. Pools of particles references a group. Particles are only unphysical objects that allows to get data in a esay way. The real particles data are stored in Group. So when synchronizing what needs to be copied is not the pool but the real particle data (which are organized well for cache optimization). If you only copy the pool, you keep the reference to the other group. The internal structure is a bit odd I admit, this is due to the fact that the engine has considerably evolved since the first version. The price is workarounds not always well designed. It works well when abstracting the internal structure however. This is why I started a version 2 that redesigns the engine from scratch for a clean and efficient internal structure to have a neat and well designed basis to get the engine evolve and complexify.

I wont extend the version 1.* to allow the update and the rendering in seperate threads as it will be deprecated soon. However I can do it in version 2 to allow to set up multithreading in an easy and robust way.

If you really want to test out multithreading with the current version, I can rewrite your swapRenderData method so that it works.

Juff wrote:

Hi, spark was not designed to be used in seperate threads atm. With your method, you actually render particles with the data of the groups of the updates thread. If it works it is simply a matter of luck because update and render dont happen concurrently.

Why is that ? This is due to the way particles data are stored internally. Pools of particles references a group. Particles are only unphysical objects that allows to get data in a esay way. The real particles data are stored in Group. So when synchronizing what needs to be copied is not the pool but the real particle data (which are organized well for cache optimization). If you only copy the pool, you keep the reference to the other group. The internal structure is a bit odd I admit, this is due to the fact that the engine has considerably evolved since the first version. The price is workarounds not always well designed. It works well when abstracting the internal structure however. This is why I started a version 2 that redesigns the engine from scratch for a clean and efficient internal structure to have a neat and well designed basis to get the engine evolve and complexify.

I wont extend the version 1.* to allow the update and the rendering in seperate threads as it will be deprecated soon. However I can do it in version 2 to allow to set up multithreading in an easy and robust way.

If you really want to test out multithreading with the current version, I can rewrite your swapRenderData method so that it works.

Thanks again for the reply and detailed info.

I don't want to make any extra work for you, since you are doing plenty already. If you are already planning on supporting something like this kind of multithreading in version 2 then I'm sure that'll be perfectly good enough for me. Even just making the internal structure clearer would make what I was trying to do a lot easier. I can certainly put off doing particle stuff for a fair while longer in anticipation of version 2.

I'll just keep popping back here to see how thing are progressing in the months ahead.

Keep up the excellent work!

Hi, I wanted to talk a bit about the future interface of multithreaded system. How would be something like that ? :

SPK::System* system; // Lets say that s the system to update

SPK::SynchronizedSystemPair* syncSystems = new SynchronizedSystemPair(system);

// On the update thread :
syncSystems->update(deltaTime);
// or equivalent
system->update(deltaTime);

// On the render thread
syncSystems->render();

// To synchronize (in any of the threads supposing neither update or render is performing on threads)
syncSystems->synchronize();

Where the SynchronizedSystemPair is an object that holds 2 systems.
The second system (the render system) is totally encapsulated within the systemPair object and constructed from the system passed in parameters with the minimum stuff for rendering. Calls to update and render calls respectivelly the update and render methods of the correct system while synchronize copies needed data for rendering from the update system to the render system.

Juff wrote:

Hi, I wanted to talk a bit about the future interface of multithreaded system. How would be something like that ? :

SPK::System* system; // Lets say that s the system to update

SPK::SynchronizedSystemPair* syncSystems = new SynchronizedSystemPair(system);

// On the update thread :
syncSystems->update(deltaTime);
// or equivalent
system->update(deltaTime);

// On the render thread
syncSystems->render();

// To synchronize (in any of the threads supposing neither update or render is performing on threads)
syncSystems->synchronize();

Where the SynchronizedSystemPair is an object that holds 2 systems.
The second system (the render system) is totally encapsulated within the systemPair object and constructed from the system passed in parameters with the minimum stuff for rendering. Calls to update and render calls respectivelly the update and render methods of the correct system while synchronize copies needed data for rendering from the update system to the render system.

Sounds good to me Juff.

Juff wrote:

Hi, I wanted to talk a bit about the future interface of multithreaded system. How would be something like that ? :

SPK::System* system; // Lets say that s the system to update

SPK::SynchronizedSystemPair* syncSystems = new SynchronizedSystemPair(system);

// On the update thread :
syncSystems->update(deltaTime);
// or equivalent
system->update(deltaTime);

// On the render thread
syncSystems->render();

// To synchronize (in any of the threads supposing neither update or render is performing on threads)
syncSystems->synchronize();

Where the SynchronizedSystemPair is an object that holds 2 systems.
The second system (the render system) is totally encapsulated within the systemPair object and constructed from the system passed in parameters with the minimum stuff for rendering. Calls to update and render calls respectivelly the update and render methods of the correct system while synchronize copies needed data for rendering from the update system to the render system.

Hello,

Just popped back to ask if there had been any progress on this particular part of SPARK 2, support for putting the particle renderer and the particle update onto two different threads?

Hi, after having thought a bit about it. I think the best way to use particle systems in a renderer with a multithread model update/render is to perform both the update and the render in the render thread. Because the dynamic nature of the impose a copy of all the data of the particles. Therefore either you ll have to synchronize your threads for all the time the copy takes either you ll have to double buffer the data.

As particles will in general not interact with your logic and are visual effects, having them totally on the render thread makes sense. On your logic thread, you can use a kind of dummy particle system object that will be used to communicate with the real system on the render thread. For instance to update the transform of the system or whatever.

Copying the whole dynamic data of particles is not worth it to me because the time it will take to copy/synchronize may be greater than the speed gain of using concurrency.

I havent work on concurrency yet as I have spent the little time i ve got for SPARK on tasks with more priority (such as serialization)