Tag Archives: Video game

There’s a lot of photogrammetry apps out there for 3D scanning the real world. Those techniques could also apply in the virtual world. For example, by taking screenshots of a scene in a video game, it should be possible to extract a 3D model, without decoding the native files.

Here, I 3D scanned Harvest Moon: A New Beginning (Harvest Moon is one of my favorite video game series).


Extracting a scene from Harvest Moon: A New Begining is perfectly challenging. The game is played on a Nintendo 3DS, which has no native connections to a computer. To get the native 3D models, (I’m not even sure if this is possible), you probably would have to reverse engineer the ROM/game data. Luckily, you could install a homebrew 3DS video capture board onto the 3DS to output video to a USB connection. The outputted video could be used for 3D scanning!

The game also has a 3D world with some camera rotation movement. Camera rotation is crucial because otherwise there would not be enough information captured by the images. The game is also shaded in a cartoon-like way. This style of shading (which I love) will pose challenges to 3D scanning, which is expecting real-world light/color conditions.

Unfortunately, I don’t have a capture board. I could theoretically use my camera to take photos of the screen, but there would be a lot of noise. The next best thing was a Youtube video I found (by Thomal9) that had 2 seconds of footages I could use (Youtube video: time 7:41-7:42).

Harvest Moon Scene

Above is one of the images I used for my 3D scanning. I used a total 12. They are not high in resolution.

I used VisualFSM and Meshlab, and followed Jesse’s Open Source Photogrammetry guide and got the following result with the default Poisson surface reconstruction (the same in the gif up top).

3D scanning result

*One note about using VisualFSM on Windows, which differs from the instructions, you need all the files from the CMVS-PMVS folder for Yasutaka Furukawa’s patch-based dense reconstruction, and not just pthreadVc2.dll.

alternate reconstruction

Here is an alternate reconstruction using 4 for both octree depth and solver divide. The house is much better formed but the detail of the mailbox and the farmer are complete lost.

Given the low resolution of the screenshots and the lack of diversity of the scene, the 3D scanned model of Harvest Moon: A New Beginning came out pretty great. As a proof-of-concept, it shows that 3D scanning virtual worlds is viable. With a little cleanup, these 3D scanned models can be 3D printed, to create custom figurines when the native models aren’t available.

In the future, I’m going to try 3D scanning a high-resolution native PC game to get better results.

Update: Better results with the game ArcheAge

[Edit: Full tutorial in MAKE: Vol 38!]

Video Game Plushies

Meet Max. He’s a Baby Roshan, from the video game DOTA2.

I algorithmically generated his sewing patterns from the 3D model, custom printed the fabric, and stitched him together.

How I made Max:

I started with this awesome 3D model of Baby Roshan. It is made up of approximately three things.


The first is a set of 3D points, to define its shape.
3D points

The second is a set of 2D points (called UVs), to define how textures are applied to it.


The third is the texture itself.

I took that model, and created a new set of UVs for it, such that whenever two sets of UVs joined, a seam will be created on my final plushie.

custom UV mapping

new UVs

Given that the two sets of UVs map to the same 3D model, transformation matrices can be calculated to transform the old texture into a new texture. Using scripts that I wrote, I transformed every face, and assembled the result. Here’s one of several pieces.

Piece 1

This new texture I printed into fabric and sewed together to create Max.



I hope this inspired you.

[Full technical/mathematical explanation of what I did here]

I was ecstatic the day my friend gave me a DOTA2 beta invite. Having been a fan of the first DOTA (Defense of the Anicents) game, DOTA2 filled me with warm nostalgia.

It also filled all my free time. For the past three months, (as semi-evident by my lack of posts and spiffy projects), I became entrenched in the world of the Radiant and the Dire, ganking enemies, buffing allies, causing general ruckus. I also met a lot of awesome people like one of my friend’s internet posse. We team up nightly for epic games.

When Valve brilliantly released their DOTA workshop, allowing user-made models to appear in-game, I was thrilled. I’d dabbled in 3D-modelling as a bright-eyed youngin’ and was super interested in building custom gear for some of my favorite heroes.

My first time in the workshop, I made an amazing discovery:


They kindly offered models of the DOTA heroes in convenient formats, which inspired me to 3D-print mini-figurines.

Behold my adorable mini-Furion (Valve calls him Nature’s Prophet):



He’s a cute 1.5 inches tall with all sorts of nifty details that my phone camera cannot capture.

Currently he stands guard at my desk with a mini-keychain Crystal Maiden (with a giant head hole for threading and everything!).

Crystal Maiden

Using Sculpteo to print, the total cost for printing (and shipping) was $15.20. Crystal Maiden came for free, thanks to their nice folks at giving away keychains.

How-To 3D-Print Your Own Mini DOTA2 Hero:

  1. Download a model from Valve’s DOTA2 Workshop. Picking a good hero to 3D print is tricky. A lot of heroes lose their personality when untextured. Also, a lot of heroes have tiny components which are too fragile to be printed.

  2. To 3D print using commercial services like Sculpteo, the model needs to be in an OBJ file (which comes with some heroes) or a STL file. Since all heroes come with a Maya file, Maya can be used to export the heroes into STL (only with newer version of Maya) or into OBJ (any recent version of Maya). Heroes can also pose, by moving the various skeletons. I made my Furion hold his staff.

  3. After exporting into the appropriate formats, the model needs to be checked to see if it conforms to a manifold (Shapeways has a nice, simple article on manifolds pertaining to 3D printing), has holes, has inverted normals, and has other similar non-printability problems. A lot of commercial services will do that check automatically upon upload. So will netfabb Basic.

  4. I uploaded my Furion to Sculpteo, scaled it accordingly, and hit print, and 3 weeks later, VIOLA! Sculpteo offers a really nice check unprintable thin components and a real-time scaling for price adjustments.

[edit: tutorial for 3d printing in color]

One of our friends gave my roomie and I a spiffy tour of Demiurge Studios.
It was super nifty to walk around a game studio, and see part of their daily bustling.

..though, I’m not sure how much daily bustling we saw, ’cause it was THURSDAY GAME NIGHT!

While we ate from the Mount Everest of pizzas, we played all sorts of games from their gigantic collection.

Thursday Game Night
I met all sorts of awesome folks like the artist behind the bathysphere of Bioshock!

I descended into Rapture with that thing!!!

Our exciting night concluded with us testing Demiurge’s game in development.

Though I’m not a console FPS gamer by any stretch of the imagination, their game was incredibly fun.
I had a great time.

What would riding a mutalisk feel like?

Probably not very comfortable. They’re a little spiny.

Now, ultralisks, that’s the way to go.

Nice and slow and steady. A little hard, but nothing a good cushion can’t fix.

ultralisk ride

They also got those nice kaiser blades, perfect for scaring off that mean driver who cut you off in the street or backing up your awesome buds in a tense game of Starcraft 2.

Lately, my friends and I have been waging interstellar warfare. They’re wrecking havoc on lands like Ulaan Deeps while I, newer to the fray, meander in the back, frantically trying to macro and micro.

One day, I’ll be cool space warrior like them.


Until then, I gotta spawn more overlords.