Notgames Forum

Programming in code is counter-productive for people with art-sided brains. The solution to this problem exists: graphical programming. But the people who need to implement this solution happen to be its worst enemies. Because to engineers, code-based programming beats everything.

Until somebody somewhere starts believing artists when they say they want to program in a visual language, or starts realizing that giving access to artists is the best way for a creative technology to continue evolving, I find myself settling with inferior designs. Because I cannot express myself adequately in code, I need to change my ideas, I need to talk about simpler things in a simple way.

It's like someone is forcing me to write poetry in French. French is a great language. And people who are familiar with it can write beautiful poetry. But I speak Dutch. My Dutch poems are subtle and sublime. In French, however, all I can write are nursery rhymes.

I love you, Unity3D (http://unity3d.com/), you're a wonderfully well designed application. You're the best out there. But your code-based programming is ruining my art.

I couldn't agree more. I love applications like Max msp, pure data, and reaktor for making music. They allow me to create sounds and sequences that wouldn't have been possible without the visual programming, and they don't make me deal with ridiculous syntax errors. I truly wish they had viable options for this in game design (Quest3d looks swell, but its price and not being cross-platform turn me away). I mean imagine how many more people would get into game design if they didn't have to deal with code. Not just artistic minded people, but just different types of people with new and refreshing ideas. Look at the program Alice. It's a visual programming application designed for teenagers/young adults, specifically females. Now I don't know if it actually worked and their community is full of young women or not, but imagine all the different worlds/games that would be created if more things like this existed. We'd have more people that could really push the boundaries of games into new directions.

Don't get me wrong, I enjoy writing code on occasion because it makes me feel smart, but after awhile the repetition and the errors I make really drag on me and my creativity.

Being, to some extent, one of those boring engineering guys myself, I can still understand the need for more graphical programming. However, even if I, and many others, understand the need for it, I would still be unable to do such a system. Mainly cause I can't visualize how such a system would work. I would mainly think of how to visualize and represent the code graphically. But that's not what we want. A little box representing an if-case is still an if-case, despite its graphical representation. We need a more fundamental change, which I'm afraid programmers just won't be able to do. Not necessarily because they think it would be stupid/unnecessary or think their way is always the best.

A condition statement is probably the worst example you can give against direct visualization of code, as people are already familiar with its visual representation.

(http://www.2oceansvibe.com/wp-content/uploads/2009/02/problem-solving-jason.jpg)

Yeah, I just wrote the first thing that came to mind. I hope you still understand what I mean, though.

This is a really interesting topic since I'm currently trying to design a kind of programming environment that works exactly the way I would like it to work. It will probably not be completely visual though, since while I do love the idea I have never seen an implementation that scales well. Gamemaker has got a pretty nice one for example, Unreal too it seems (haven't tried it hands on). But when things are getting bigger and hairier the visual clutter removes a lot of the benefits, I think. Anyone got a favorite example to prove me wrong?

3DVIA Virtools (http://www.virtools.com/) / 3DVIA Studio (http://www.3dvia.com/studio/)

But I agree with Chainsawkitten, a mere visual representation of code isn't enough. Personally I believe a virtual holodeck type of environment with a theater style of production flow would be a good starting point.

I hear very good things about Max/MSP but haven't used it myself. Given, however, that my partner Auriea -who is a lot worse with code than I am- created several applications with it, it's probably very good.

I have used Quest3D a lot myself. There is an "IF" block in Quest3D. It takes a value as input and when this value is more than 0, it calls whatever logic is connected to its output. Now, this may be a very literal graphical representation of code, but I swear it makes life so much easier for people like me! To simply see the logic I am programming rather than having to interpret text, makes a big difference. And then I'm not even talking about the Finite State Machine in Quest3D. Remember those diagrams that you draw when you want to program complex logic? Well, that's what you do in Quest3D. Except that, when you're done drawing, the thing actually works.

I'm sure that such simple, literal visual representations of code, are only a few iterations away from being revolutionary. So, dear engineers, don't be shy!  :)

Now if someone would combine the core concepts of Quest3D flowchart programming with the design cleverness of Unity3D...

http://forum.unity3d.com/viewtopic.php?t=31132 ;)

These look interesting, but I'm totally baffled by the website and what each of the different tools are for.

You're not alone, their communication / marketing has always been poor at best. Anyway, what you need to know is ..

- 3DVIA Studio is the ( spiritual ) successor to 3DVIA Virtools
- 3DVIA Virtools is expensive, 3DVIA Studio is free*
- 3DVIA Studio is currently in Public Beta

*3DVIA Studio Pro won't be free, similar to the Unity Indie / Pro pricing structure.

And no, I don't use their products ( anymore ), but I do still think Virtools has the best visual programming interface to date.

Ahh thanks, it's much clearer now, I might have to try it out.

I am totally with you on this. A while back I posted some thoughts about this on my blog (http://evolutionlive.blogspot.com/2009/03/google-is-future-of-games.html), and thought I don't know yet what the solution will be, exactly, I am on the lookout for projects that do it right and for ways that I might be able to make it happen myself.

It seems to me that every tool I've found just takes normal programming and makes it slightly more visual - the whole issue of "A little box representing an if-case is still an if-case, despite its graphical representation." that Chainsawkitten mentioned. As someone who is fairly fluent in programming myself, these tools just make the process more awkward with addressing much of the fundamental problems of this approach to software creation.

Though it is interesting, Michael, that even this small change from a linguistic to an iconic representation of programming logic is enough to make a big difference in accessibility for you.

I'm curious to see how LittleBigPlanet 2 (http://www.youtube.com/watch?v=n3sk35a8U6A) delivers on its promises of something along these lines. Not getting my hopes up, but I'll read the reviews when it comes out. ;)

As someone who can approach the issue from both the artist and the programmer side of things, I feel as though maybe I have a particular obligation to contribute to a solution to this problem. I know my artist side would really appreciate that! ;D


Cool! I'm curious to hear more about it. :D Can you share any details about it here?

Yay, glad you're interested! It's just in my head/notebooks still so nothing is final. All I know that it is going to be very visual, but probably not in the flowchart kind of way. More like "climbing" around in the code if that makes any sense :) I'll post about it here as soon as I have something to show!

This is what I found when looking for examples of Max/MSP:

(http://www.benjaminyobp.com/blog/wp-content/uploads/2010/02/maxpatch.jpg)

Yikes! It seems like a lot of these interfaces are bad at encapsulating things; all the code is in one big panel. Just being able to create your own box with the connectors you need and then hiding its contents seems like an obviously good idea. But that must be possible in at least some of these visual languages..?

Virtools does exactly that ( and does it extremely well ).

Ok, cool. Thanks for the info, will try to have a look at it!

Good luck with that (http://www.virtools.com/trial/) :P

Heh, I watched a youtube clip instead. Looks pretty neat.

I find this thread interesting, I very much switch between artist and programmer and graphical ways of programming make me climb curtains. I remember using Virtools and having stacks of boxes for what would have been 5 simple lines of C++.

But I did program games for many years now so I have now adjusted to it... to me, programming is part of knowing how to use a hammer. It will take ages upon ages to do anything functional, but the same goes for all production for games.

Part of me just wants to mumble that being able to program is much like being able to use a hammer if you want to make statues - but I would be more interested if someone could provide some storyboard or script as to how they imagine the right interface for them would work. There ought to be some space for making things like that, even if it would be beyond me at present.

Do you just want a graphical representation of what code would normally do (which is Virtools) or more of a semantic interface?

Argh, after some research it seems like someone has already started working on what I was planning to revolutionize the software industry with ;) http://www.youtube.com/watch?v=PsPX0nElJ0k (http://www.youtube.com/watch?v=PsPX0nElJ0k)  I hope they at least don't patent it because I want to use very similar ideas but in a more beautiful and less "corporate" looking way.

Anyway... in a way I totally agree with you Joeren, code can be very elegant and to the point. Expressing "x = 5 * cos(t)" or something with nodes is not going to be an improvement. Also, almost all programming languages lets you define your own words so that you can do things like "if(lion.angry) then human.runAway()" which is very powerful since it reads so nicely. Tracing lines across the screen just doesn't seem as good.

But I also think that code as it looks and works today is definitely sub-optimal for most people and I think it's great if we can think of how to improve upon it. Sounds like a really fun challenge to me! (and would definitely be good for the notgames cause)

There's a lot of text in that video, Erik!...  ;D

Please don't underestimate how powerful and enabling a "straighforward" graphical interface can be. This is not about reinventing the wheel necessarily. The "spaghetti" of the Max/MSP screenshot is just an example of sloppy work. But some people like this. In Quest3D, you can collapse branches of the flowchart in folders. Or you can simply make hundreds and hundreds of different flowcharts that all call each other with parameters, etc.

This is what part of The Path looks like in Quest3D (this makes a character walk to a swing and sit down):
(http://tale-of-tales.com/blog/media/ThePath_in_Quest3D.jpg)
This works from top to bottom, from left to right. The entire flowchart is called every frame when running, starting with the orange box at the top.
The dark blue boxes are callers to other flow charts, called "channel groups" in Quest3D. The orange box at the top allows for other groups to call this one.


It doesn't matter that it is more cumbersome to connect two "value" boxes to an "operator" box instead of simply typing "5+4". The point, for me, of graphical programming, is not to make the work faster, it's to allow me to be more creative, to experiment more, to "paint with algorithms". Even if it takes more time to set up, and is not as elegant, the fact that I can see the logic makes all the difference to my brain.

When I look at the flowchart above, I see how the logic works. Without having to decipher any code. Maybe this is different for engineers. But my brain is happy when it can look at flowcharts instead of text.

You're right, that's why you use a calculator block :)

(http://img265.imageshack.us/img265/3660/calculatorw.png)


You ( can ) still have that in visual programming obviously.

(http://img831.imageshack.us/img831/5881/condition.png)

Personally I think the whole concept of programming is too peculiar for outsiders regardless of presentation. A tool that enables anybody to create a video game would have to be closer to human "language".

I would say that code, can be very intuitively read if you use lots of OOP to make it more like sentences (as Erik said) and also if you use an IDE which colours keywords in nicely for you (this makes things a lot more readable or me). Although there will definatley be times when more graphical methods are easier to use, such as long if-else blocks... kinda like in that snippet from the Path's program. I guess everyone beat me to the key points.

And "paint with algorithms". That's a great phrase, you should use it more often. I would use it all the time, but I don't think I'd ever have call for it. I also didn't really see why you'd want a graphical interface until you said that. I was thinking how it would be exactly the same but just with boxes instead of code, then that phrase gave me an apithany (kind of), as though programming were like writing a script for the program to follow, while doing it graphicly is more hands on, more like making/shaping something.

That last paragraph should have had more of a point. Somehow my thoughts always get lost in translation between my brain and my mouth/writings.  :-X


I'd come across this website before but never figured out why they had so many products which all seemed to do the same thing. Now, finally I understand. :D

haha don't worry, happens to me all the time.

Code bubbles looks intriguing. I like the way you can use it to re-arrange the code and how it connects multiple bubble to an adjacent one. It also seems much cleaner for analyzing segments of code and editing it, but is that all it's for? I mean, do you have to import your code into the IDE first and then open them into the separate bubbles so you can further work on them? That's what I got out of it anyway, it's a really cool idea and allows for much cleaner programming, but I don't see me using it. Hopefully people expand on this idea, like Erik, and make into something marvelous.

Yeah, "code bubbles" is a lot of text. But at least it's presented in a way that emphasis the flow through the code in the same way flowchart-code does. I think its a clear improvement. If you chop up your functions into really short ones (something that code gurus often encourage) you'd get almost (sorta...) the same look as Quest3D (which uses text inside all boxes it seems).

Very interesting to see the "The Path" code. I like that I can remember the shape of the tree and easily go back to places I've "visited". It's pretty hard for me to understand exactly how it works though; for example "if -> A -> Ending?" at the top seems very cryptic :P

Both "painting with algorithms" and "programming with natural language" seems very intriguing. I know that the program environment Director used to have some kind of programming language that kinda looked like normal english, but they seem to have changed it into something boring c++looking crap. Here's a link: http://en.wikipedia.org/wiki/Lingo_(programming_language) (http://en.wikipedia.org/wiki/Lingo_(programming_language))

But then of course there's also this discouraging quote:

"Projects promoting programming in "natural language" are intrinsically doomed to fail." -- Edsger Dijkstra

It seems as though programming in "natural language" would not only be hard to create but also, I think it might even be harder to learn than regular coding. There's too many multiple ways of saying the same thing, not only replacing certain words with others, but using a completely different sentence, as well as the many different dialects that people use. It would probably make you relearn your own language, certain habits and sentence structures, in order to make it work properly/efficiently. I don't know that's just what I think.

@ Erik & badvibration - Agreed. I didn't mean "language" ( hence the quotes ) in the literal sense, but in the way a teacher guides a dancer, or kids explain a game on the street as they go.

An "interesting" thing would be if ambiguity in the language could be executed in a non-deterministic way. Of course it would be hell to debug but it could maybe lead to a more organic feel with opportunities for discoveries and "play", like Michaël was talking about.

OK. Do you have any picture of how that "language" could work/look like, practically speaking?

Somewhat similar to how 3D applications such as Maya / Softimage let you work on the end-result ( or at least significantly closer compared to code ) in real-time while hiding all the vertex shoving, matrix popping / pushing, quaternion lerping etc. In fact, most artist aren't aware of what exactly goes on behind the scenes ( and most of the time they don't really need to anyway ).

Good points.

Maybe programming a game should be more like instructing actors? With invisible "stage workers" for things that should go on behind the scenes.

So I've been thinking about this a lot over the last several days. Actually I've been obsessively thinking about it non-stop and reading everything I can on related topics online.

I tend to do that for a different thing every week. This week, it's been this.

So there are a few pieces I've been focusing on, that seem most crucial to the success of a programming or game development tool for artists. There are probably others, but I thought I'd share what I've been thinking about so far.

One is readability through R-mode (http://www.drawright.com/theory.htm) perception.

First of all, a disclaimer: When I say "R-mode" versus "L-mode", as in "Right brain" or "Relational" or "Rtistic" versus "Left brain" or "Logical" or "Linear", I don't mean to suggest that the brain is really divided into strict differences between its physical right and left halves. That is an outdated belief. But I find the terminology to be a useful shorthand.

Thinking on Michael's comments about visual flowcharts being easier for him to read than linguistic code, and looking back on my own experience, I think there really is something significant about how the code is presented and perceived, even when the underlying logic is the same.

When I am reading code (or a book!) I am usually using what I call "L-mode" perception - going through in a linear, linguistic way, and building up my mental model one step, one line of code at a time, following the logic that is expressed symbolically, in sequences like that.

However, sometimes my mind is in an "R-mode" of all-at-once, spatial perception like you'd use for looking at a painting or trying to find a certain LEGO piece in a big box of pieces. When I am in that state of mind, and I look at code (or to a lesser extent, written language) I see all the words at once and perceive the spatial relationships between them, and the underlying logic of the code is utterly incomprehensible to me. Obviously not the "right" way to read code.

But maybe it could be.

Ha, that would be a good tagline. "The right way to code." :P

The thing about R-mode perception is that it's a lot easier to be creative when you're in it. The other thing about R-mode perception is that artists are usually a lot more skilled at functioning in R-mode than they are in L-mode.

Therefore, if you had a tool that let you do programming while in R-mode rather than L-mode, it might be slightly easier to do creative things with it. At the least, there would be less inefficiency caused by switching between R-mode and L-mode whenever you think about what you want to change and then have to dive into the code to actually change it.

However, this may not even be possible.

All the visual programming editors I've seen, all the examples that have been posted here, require an uncomfortable mix of L-mode and R-mode perception in order to use. What I tend to see is a bunch of visually identical boxes connected by lines, and differentiated by text.

What you see in R-mode is the set of relationships between the boxes. But you can't tell what each of the boxes does. To do that, you must read the text and think symbolically, in L-mode. Really, very little information is conveyed through spatial relationships, through R-mode. Most of it is still sequential and symbolic.

For that reason, I find that pure written code, all L-mode, is much easier for me to deal with, since I don't need to switch around multiple times a second just to figure out what everything means. However, I suspect that there may be a way to create a pure R-mode method of programming too. But I'm not confident that it's actually possible. Just intrigued enough to try.

There are some programmers who hate the idea of visual programming, and say that it's a waste of time to use spatial relationships to convey the meaning of code. If you are one of those people and you use syntax coloring or indentation, you are a hypocrite.

So there's one aspect. Make sure your tool is R-mode accessible, if you want artists to be able to use it.

The second thing is building with functional pieces in real (or almost real) time.

Artists tend to appreciate tools where "what you see is what you get" - you're manipulating the end result, so you can immediately see the results of your actions. The process becomes more like sculpting.

Programmers tend to discount such tools as nice but unnecessary. They are used to typing in code for an hour, hitting a button, and waiting a minute for everything to compile and show up on the screen.

These are two fundamentally different mindsets, as different as a slideshow and an animation.

When you operate in the slow, "slideshow" approach, development and creativity tends to happen in an architected, "top-down" way. You have a plan, which is in your head, and then you put in a bunch of time and hard work to mold reality into the shape of that plan.

There is a fundamental shift that occurs as you decrease the time between action and result. It's as real as the shift that occurs when you hit 24 frames per second - from slideshow to animation. To your brain, it's alive, it's moving.

When you operate in the immediate, "animation" way, development tends to operate in a more exploratory, "bottom-up" process. You don't have to have an entire plan in your head. You see the results of your actions immediately, and if they are surprising or unexpected, you can adjust your plan. You can try random things and follow them if they prove to be interesting.

In the area of game design, innovation is much more likely to come out of an exploratory process than an architected one. As Jonathan Blow said earlier (http://braid-game.com/news/?p=666). It's hard to do things that haven't been done before if you have to plan it all out in your head first.

So we want a tool that allows us to sculpt the end result, with immediate feedback.

Part of this is that everything you can make should work. It may not work in the way you desire or expect, but it should still do something.

If you are painting with pixels in an art program, no matter how you put those pixels down on the screen, it will always be a functional, viewable image. It might not be pretty, but you can still see it. You are never going to run into a error message that says, "Invalid pixels at position 55, 46. Image cannot be displayed."

But if you are writing the code for a program, this sort of thing happens all the time. Most of the things you can type won't work at all. They won't turn into a program, even a broken one. There are right ways to write code, and wrong ways to write it.

I would say that this also makes a big difference. Perhaps the biggest difference is that writing code has a much higher barrier to entry, more learning how to do things at all before you can start learning how to do them well. But it also makes experimentation so much more difficult. You can't throw a bunch of random stuff together just to see what happens. Because what happens is nothing. It just won't do anything at all.

So if you can build only with pieces that work, and immediately see what changes, this would make truly artistic interactive art much easier to create.

The last thing is expressing general logic through specific examples.

This is probably the most impossible and most revolutionary but least important of the three. If you just had a tool that you'd use in R-mode, that let you shape the end results with immediate feedback, that could be awesome, and probably enough to make a huge difference.

But at the same time I am intrigued by this further vision I have of providing specific examples, which the system will extrapolate to create possible general rules for creating those example, which you will then provide feedback on and refine in order to guide the system's hypotheses toward the end you have in mind.

Because I don't see how to actually avoid symbols when describing logic, or how to directly manipulate end results in a general way. Because games are systems, and the end results happen when you take the rules that you have set up and run them through their paces.

So maybe this is the only way to achieve those first two goals in their entirety.

What am I talking about?

You know how you draw diagrams and mockups for different things that happen in different situations in a game? Like this (http://www.lostgarden.com/2007/09/tree-story-wireframes.html). It's a pretty common way to organize your thoughts when you're designing. The thing about those is they're all organized around specific examples, not general rules. So you might draw a diagram with a guy hitting a wall, showing how he bounces off or breaks through it or whatever. It's not completely specific, as you might have an abstract line standing in for any kind of wall, and a stick figure representing any kind of guy, but at the same time it's very concrete. And you can add general connotations by writing in little notes, to explain the rules behind the example more clearly.

The reason that we don't just stop there is that our game development tools require everything to be spelled out exactly - they cannot extrapolate from these examples, because there is so much ambiguity. It could mean this or it could mean that.

However, we run into a similar problem when trying to communicate our ideas to other people who are helping us make them into a reality. Especially if we are designers and we are telling programmers what to do. How do we solve this problem with other people?

Part of this is by clarifying with more examples when an area is unclear. Kind of starting at the highest level and breaking it down into more specific situations when necessary. Another part is through conversation, asking "It sounds like you're describing this... Is that right?" and responding "Yes, exactly!" or "No, I was thinking something more like this..."

Both of these could be accomplished with a special computer program instead of a human programmer. Maybe not as well, especially in terms of accuracy of translation, but in some ways better - particularly, in the time between your description of a design and seeing something on the screen. And this increase in speed could make up for the lack of accuracy, since you can adjust and correct much more quickly. And as a result, make use of exploratory design instead of architecture.

Break dynamics down into stories instead of rules. A playthrough of the entire game could be an example story, and you could create example stories of successively smaller and smaller pieces of the game until you have specified it completely. Or completely enough.

The tool generates possible rules that could create the situations you specify, and presents several for you to try out. Most likely none of them work the way you want. Pick the one that's closest, and let it generate more possibilities based on that. It's an evolutionary search. Like Biomorphs (http://www.eccesignum.org/flash/bio/20040611/).

And stories don't always have to be specific stories about specific instances. They can be more or less abstract and universal. Like Raven, with a capital "R", who is both the character Raven and all ravens and all tricksters at once. Or the princess in the tower, or the wise old man, or the dragon in the cave. Or the stick figure on the crosswalk sign who represents all pedestrians who could ever walk this way. There is a continuum between the specific and the symbol.

I am particularly inspired by the concept of the Dreamtime (http://en.wikipedia.org/wiki/Dreamtime). The translation of this name is misleading, as it does not refer to a time in history. It is like a parallel slice of the world running alongside and underneath the specific, physical world, where the gods and heroes walk, creating and personifying the dynamics and processes that underlie everything we see in reality.

I want a tool where I can not only shape the world as a level designer, but also shift into the Dreamtime and shape the dynamics of that world as concretely I would shape the placement of coins and mushrooms.

When this happens, we will get our interactive art.

Update:
Just reposted on my blog (http://evolutionlive.blogspot.com/2010/09/how-artists-want-to-make-games.html)...

Wow, that's a lot of interesting thoughts Axcho!

I can see how you think this would work and it is an interesting and cool idea. Very hard to implement though... I'm trying to imagine how someone could make a very small "proof of concept" but it's hard. I'm thinking a lot about these things right now too, I will get back as soon as I have some ideas.

That's awesome axcho. :)

Here's a starting point.  You have a box in a physics environment, and you can change parameters for it visually.  If you want to change the bounciness, you can move the curve that predicts the movement of the box.  Want to make it less bouncy?  Lower the top of the first arc after it hits the ground.  Like this:
(http://www.physics.hku.hk/~phys0607/images/soc_18.jpg)

You could drag the second arc here up and down.  The second image is the result after you drag it down.

That's just one parameter.  Another could be drag, which could be adjusted by moving one of the first two circles in this image:
(http://farm3.static.flickr.com/2074/2389444869_d0f16fd9a6.jpg)

Speaking of "natural language", I always wondered what a programming language would look like that was based on my own mother tongue, Dutch, instead of English. And would it give Dutch-speaking programmers an advantage?

That's a great start, God at play.

And Michaël, please give us an example of a programming statement in Dutch :D

Here's one in Swedish, just for fun:

Flytta karaktären snabbt framåt (på ett snyggt sätt) när jag trycker på uppåtpilsknappen, tack.

Hey, glad you like the idea, thanks! :D

Starting with a physics simulation could be a good idea for a proof of concept.

Here's the image that first got me thinking along these lines:

(http://img.skitch.com/20090902-c61gwx497chf318ekchrjtia5h.png)

This is brontosaurus (http://brontosaurus.deviantart.com/) showing the process behind his Tree Test (http://brontosaurus.deviantart.com/art/Tree-Test-135828654) drawing, so I could make a procedural version of it.

(http://fc05.deviantart.net/fs47/f/2009/247/9/9/99921deea1126c65ab299e0f0aa8409e.jpg) (http://brontosaurus.deviantart.com/art/Tree-Test-135828654)

I never got around to it, but the interesting thing is that he didn't have any specific rules in mind, just some implicit guidelines expressed through these examples. The idea was for me to get inspiration from the examples in order to code up specific rules that would create similar patterns.

Procedural pixel art like this could be another interesting avenue for a proof of concept. :)

The step-by-step nature of it I think would be essential.
Stories, sequences. Like an IQ test for the computer. ;)

That codebubbles thing looks pretty interesting: all the structure of code, but with some more 'natural' (I couldn't think of a better word) arrangements/flow. It looks like the sort of thing I would find very useful while coding.

(It took me a while to realise that it's easier to work out what's going on in that video with the sound on)


@axcho
Some interesting ideas. I can almost picture it, it's like one of those things you can just about see, but can't quite define. The videogame creation equivalent of playing an instrument, how dreamy  :D


???
I'm glad programming isn't in sweedish, that looks very complicated (probably because of my complete and utter lack of familiarity with Sweedish or any similar languages).

So I'd say English speakers probably do have the 'native tongue advantage' when it comes to programming.

I do wish that the boxes were a little bit different. Though the colour coding of called, public and parameter boxes in Quest3D helps. But you know, also the little differences do. Some boxes have 1 child connector, others two or three. So even that small difference helps me to tell them apart. That plus their context. I have a personal way of making layouts with the boxes that helps me recognize an if-else-then structure very quickly, e.g. So, while it may not be easy to tell the difference between the different boxes, the different layouts of multiple boxes are much easier to differentiate. It's like making simple drawings, almost. Also, Quest3D allows you to add coloured boxes and pictures to your flow charts, which can also be very helpful.


I agree with your desire for a completely R-Mode environment. But I really don't want the "utopian" nature of this desire to hold back the very real possibility of graphical programming systems. Quest3D, while imperfect, is infinitely more fluent for me to program in than any code-based environment. Despite of being an artist, I have used it to create a multiplayer game, an reusable AI system and a rich game like The Path that has a sophisticated music system, dynamic graphics, autonomous behaviours, etc. There's no way I could have made any of these in Unity (because Unity's programming is script-based)!

I also agree with the real-time aspect. Again, as opposed to Unity, Quest3D is real time: the game is running all the time, as you program. The entire flowchart, in fact, represents a single frame in the game, a frame that is repeated over and over. Your flowchart simply contains switches to decide which part of the flowchart should be called when. All this time the game is running and in the flowchart, the boxes that are being called light up. Being able to change your code while the game is running is incredibly powerful for "R-Mode" design!
Unity allows for a little bit of that, thanks to being able to expose parameters in the editor's interface. But it's not the same as real real-time editing.

I really appreciate your feedback on this, as you are one of the few here with actual development experience in a visual programming language!

Even the rudimentary R-mode readability that Quest3D offers is still very helpful, it sounds like you're saying.

On that note, I had an idea to bump the visual differences up a bit more - to use randomly generated "invader" sprites for each type of node, so they are visually distinctive. They're random bitmaps, mirrored across the vertical axis, so your brain recognizes them more easily, like faces. The variations are effectively infinite and they're easy for computers to generate. Whenever you create a new type it would pop up a new, unique random image to use. Still symbolic but more visually distinctive than boxes.

Like this:
(http://fc05.deviantart.net/fs30/i/2009/248/2/8/Active_Sketch_01___Invaders_by_axcho.png) (http://axcho.deviantart.com/art/Active-Sketch-01-Invaders-135939876)

Ideally the visuals would reflect the actual logic or behavior of what is represented, but as you said, I don't want the utopian vision to hold back practical but imperfect improvements. Like making each node visually distinct. Do you have any thoughts about how this random invader icon idea could be applied? Maybe the inputs and outputs could be incorporated into the image - the more meaningful the visuals, the better.

What do you think? :)

Another interesting question is what are you actually manipulating with these visual flowcharts?

In Quest3D you are manipulating the flow of logic. Commands and if-statements and all that.

However, there are other options. Like manipulating the flow of data - dataflow programming (http://en.wikipedia.org/wiki/Dataflow_programming).
Check out this tutorial on Prograph (http://www.mactech.com/articles/mactech/Vol.10/10.11/PrographCPXTutorial/), a visual programming language for manipulating the flow of data rather than logic. There are pictures.

I wonder if that might have any application for (not)games? Maybe in a modified form, to support data-oriented design (http://gamesfromwithin.com/data-oriented-design), perhaps? I bet a visual programming language could be better suited to that paradigm (http://solid-angle.blogspot.com/2010/02/musings-on-data-oriented-design.html) than the traditional languages like C++ and such. Apparently that's where things are headed, anyway, as processors get faster and memory access times lag behind. It would be pretty ironic if people had to turn to visual programming in order to write optimally efficient code for their games!

In addition to manipulating the flow of logic, or data, there could be other opportunities for visual programming to shine. For example, representing the connections between objects. Usually the most verbose and confusing bits of code I see have to do with one object accessing data (or functions, whatever) from another object through some convoluted chain of references, with all sorts of annoying syntax and symbols needed to represent what should be a simple idea. Linear, linguistic communication was never so good at describing relationships. That's what R-mode is for. ;) I'd really like to see this problem explored with a visual programming approach for organizing objects and groups and types of objects and drawing the connections between them.

In particular, one specific place that could benefit from a visual way to see and create connections between objects is in component-based game architecture (http://cowboyprogramming.com/2007/01/05/evolve-your-heirachy/). Like data-oriented design, it's another step up (or at least away from) traditional object-oriented programming, in this case to improve modularity. And it's great, it just can get tough to deal with all the interconnections - not just between objects, but between the components of one object, the components of multiple objects trying to communicate, and so on.

Also, a video tutorial (http://www.youtube.com/watch?v=auaqZzcjl-Y) on component-based architecture if you're new to the concept.

Thoughts?

SunVox uses exactly that to label patterns. Personally I'm not a big fan though.

(http://createdigitalmusic.com/images/2010/03/sunvox.jpg)


Agreed, can't live without it anymore.

I am astounded. I thought I was so clever to think of that. And it's already been done. This is a great opportunity then - it will be much easier to see what's good or bad about the idea!

So, you don't like the random icons much? This is very good to know. What do you dislike about them - what would have to be different for them to be actually helpful to you?

I'm really curious what problems you run into with using these random icons - what problems do they fail to address, what problems do they introduce? What could be improved? What do you like instead?

Looking forward to hearing your response. :)

Random is always a bad idea when describing something. Perhaps colors would make them a little easier to distinguish. And i like words instead :)

Or even better, no words & icons at all.

Maybe "always a bad idea" is a bit strong. I would say, "Describing something with randomness pushes your communication into the realm of symbolism - further toward L-mode, in other words." If something has no connection with the thing it is describing, it is a symbol. Like words, like language. What does the sound of "smile" have to do with the actual thing it signifies? Nothing.

This, however, is a symbol, but with a closer connection to what it symbolizes: :D
In the case of emoticons, the difference between the symbol and what it symbolizes does not destroy the meaning, it just makes it more general. It gives it a more universal connotation than an exact image of a specific smile.

So...

The existing alternative to random icons is "random letters" - words. I think the random icons are still an improvement in being somewhat more R-mode friendly, in that they are recognizable from a distance when selecting one among many, without having to "read" each one individually like you would with words. Of course, words have the advantage that once you do read them, the associations evoked are usually much more memorable (and relevant!) than those of a random icon. Still, we are talking about R-mode readability here, and if it takes an extra mental dip into L-mode to actually read the word before you get that, words are useless to us.

A choice of using an icon or a label or both per node could be practical.

But you are suggesting that we use neither words nor icons.

In that case, what is the alternative that you suggest? Just using identical boxes for each node would not be an improvement, at least for me.

Are you suggesting to represent the specific properties of the node visually, then?
You could do this with the invaders - instead of generating a random bitmap, generate a bitmap deterministically from the data of the node. Though a simple binary dump of the data would probably be too incomprehensible to be useful. There may be a way to use that general idea to make a more functional visualization though.

What do you suggest?

Update:

Invaders with colors:

(http://fc08.deviantart.net/fs23/f/2009/247/f/5/f5ee1427294d47510c89b5f9f1cf46c7.jpg) (http://brontosaurus.deviantart.com/art/Invader-Tests-135929952)

When you are using arbitrary symbols, it's good to have only a few of them, so they're easy to memorize. That's an advantage colors have - you can use a limited set, and they easily group into categories (red, yellow, green, blue, dark, light, warm, cold, etc.). And they are even easier to distinguish from a distance than the invader icons.

With colors, icons, and labels, we'd be getting somewhere. Still symbolic though. How do we drill down into representing functionality directly?

Maybe getting beyond the lines-and-boxes paradigm, to something that makes more efficient use of spatial relationships between nodes, not just within?

Actually, a lot. When I say "smile" your brain instantly collects all kinds of associations with that word. Why not take advantage of that?

My suggestion ( as previously mentioned ) would be to work on the end-result directly, instead of on a abstract representation.

Right, that's the power of words (or any arbitrary symbol that you've built up a lot of associations around). They're great. But the problem is that in order to get those associations - which I agree are "much more memorable (and relevant!) than those of a random icon" - you need to go into L-mode. That's a deal-breaker.

The point is to be able to look at the screen and see everything at once, which is how R-mode operates. If you are looking at a screen full of text, you have to read through them one-by-one, linearly - L-mode. Even if you have audio for spoken language, you have to listen to each word one-at-a-time, which is still linear.

I'm not saying you shouldn't have the chance to go into L-mode for more detail - that's why I'm suggesting a combination of icons and text - but the problem we are trying to solve is how to make the logic apparent purely through R-mode, or at least as richly as possible.

Update:

As far as making internal logic more apparent visually in the nodes, here's some inspiration:

(http://farm5.static.flickr.com/4143/4859891813_44a9ce3d25.jpg) (http://www.flickr.com/photos/bjzaba/4859891813/)

If those were logic diagrams, what logic would they describe?

Richard Evans, AI programmer on Black and White and The Sims 3, once told me that he thought that in the future visual programming languages would be required to deal with the increasing complexity of programming. I often think of visual programming as the next step in abstraction. Visual programming is to C++ what C++ was to Assembly.

I agree that colours would help. I know colours help me find the right application icons on my computer quickly, even if I don't really understand what they are supposed to represent. Their location is also an important factor.

But a figurative shape is probably better than a randomly generated one. I think it far easier to learn that a picture of a duck represents an If statement than a random configuration of pixels that vaguely looks like a moon on a stick or something.

The important thing is consistency! This picture of a duck should always represent the If statement. There should, e.g., be no way for a user to customize this. It's like logos of products or companies. These days, with literacy being widespread, they don't need to represent anything anymore. As long as they are used everywhere and consistently, they will function.

I think what Kjell is saying is more like the pictures I submitted.

You work in an environment that's like a custom mode in the game executable itself.  So to define positional logic:


You could place your cursor at 100 and somehow use a command to clamp the position at 100, like a Photoshop-style guide or something.  If you had a physics engine, I guess you'd just create a static box with one edge at 100.  That'd be one example I can think of.  In contrast, icon-based visual programming would involve creating an icon, and then connecting it to another icon, and then setting an input field to 100.

Maybe what you want to look at is custom game engine tools, since sometimes they work kinda like that.  Like a level design tool that programs logic.  I used something like this for the Marmoset Engine, although the logic was obviously limited and it wasn't really designed with this goal in mind.

Yeah, I was just suggesting randomness for the functions that you create yourself, not the atomic parts of the language. For the immutable basic operations like if-statements and such, you'd definitely want a consistent, visually distinct, highly readable form. And probably color-coded too. ;)


Yeah, ultimately that kind of direct manipulation of end results is what I'd like to aim for. I was just trying to figure out a more practical intermediate step that takes the visual programming of Quest3D and bumps it up a notch in terms of R-mode readability.

But ultimately, yes, my grand vision is a tool where you work with the end results and define stories and situations and guide the system's evolution toward the realization of your idea.

While discussing this over lunch, we realized something. That visual programming environments stimulate creativity, while code-based environments discourage it. This probably has something to do with the fact that creativity is located in the right side, next to all our visual skills.

We xompared it to defining colours in a bitmap editor. A code-based interface would offer you three input fields: Red, Green, Blue, while a visual interface gives you a colour wheel and you can just click on the colour you want. The three input fields are incredibly accurate, but they are only so for the person who knows exactly which colour they want and who knows exactly how much red, green and blue it takes to make this colour. The colour wheel allows you to choose and change and fiddle until it feels right. The end result is the same.

To be able to program in code, you need to know how to express what you want in code, before you even start typing. To program in a visual language, you also need to know what you want, but not so much how to express it. Expressing it is a process of experimenting. A visual interface invites such experimentation. And I believe, as such, it stimulates creativity and probably allows us to achieve even greater results than we imagined, something a coder is incapable of. The best result of writing in code is to express your idea. In most cases, however you don't even get close. At least in my experience. While in a visual programming environment I regularly surpass my own ideas and make something that is even better than what I envisioned.

I think this is important.

I'm listening to the fifth Infinite Ammo podcast (http://infiniteammo.ca/blog/podcast-5-kyle-pulver/), an interview with Kyle Pulver, and just heard this:


Wow. Guess I should check out Klik & Play (http://www.glorioustrainwrecks.com/wiki/Klik_%27n%27_Play)...

This is a great compromise between coding out everything in advance, and expressing general rules by specific examples. You just play, and when a specific situation comes up, you tell it what rule you'd like to apply to the situation in the future.

That's awesome. Maybe this will spark a breakthrough in our thinking about this...

That's awesome. :)

Yay. :D

Programming interactively with the computer is a nice idea. Is the description accurate? Or is this how this person imagined how it the program works?

I still have to try it out for myself, but I believe the description is accurate.

If you read about Klik & Play here (http://www.glorioustrainwrecks.com/wiki/Klik_%27n%27_Play), you'll see a similar description of the feature:

"A Step-Thru Editor for absolute beginners, which runs the game, pausing it when certain events occur in the game and asking the user what actions to take"

If you want to try it out, here are the links to download and get started:
http://www.glorioustrainwrecks.com/node/197
http://www.glorioustrainwrecks.com/node/72

I don't think Klik & Play would be of much practical use in itself, given its limitations and bugginess, but I think it would be a great reference, a great source of inspiration for a new tool...

I remember this function from the "sequel" to Klik & Play, The Games Factory. Indeed, upon any new collision event in-game, you could instantly set a consequence. Might have worked for events other than collisions too, don't quite remember. I do remember you could press escape at any time to pause the game and specify what would happen in the level after that amount of time had passed since starting the level.

More importantly, Klik & Play is ancient, and has been replaced first with The Games Factory, and later Multimedia Fusion and Multimedia Fusion 2. This program is what powered Mårten Jonsson's notgame Star Sky and everything made by Swedish indie creator Nifflas Nygren, so it seems serviceable enough. So if Multimedia Fusion 2 has the same functionality, which I'm assuming it does, that would be the better choice!  ;)

I've not found visual programming to be any easier, at least from my experience with Blender's "logic bricks" setup. I have trouble with the logic of programming so whether it's lines of code or a block diagram, it doesn't matter.

I´ve been using Multimedia Fusion 2 for a year and it dosen´t have that magical feature of Click and Play you are talking about. However, I have strictly no programming knowledge and it allows me to make the games I want. It´s pretty visual, very intuitive, and can get powerful if you are skilled enough, as Nifflas proves. It could be better for sure, even more visual and intuitive, but they are getting there, Multimedia Fusion 3 is on the works so we´ll see. I would love if you could edit the code in real time like in Unity...

Kay, that's sad. The Games Factory may be better than Klik & Play and still do these things, but I doubt anyone would call it a worthy game development tool today. So yeah, these functions would be more of an inspiration than something to actually make games with.

That only works in very rare circumstances. Most of the time a running game starts misbehaving if you change the code.

In Quest3D, on the other hand, this works perfectly! In earlier versions there was no separation between edit and play mode, even. That combined with its visual programming (which I do find essential and a million times more efficient than scripting -even if its is not necessarily easier) allowed us to make our two most complex games to date (The Endless Forest and The Path) and an alternative AI system (Drama Princess).

A new plugin for Unity, called Antares Universe, allows for real-time visual editing as well. It's still in beta but it works better than the scripting for me. We're making the prototype for the new version of 8 (http://Tale-of-Tales.com/8) with it.

It's probably worth noting that the only reason the "shoot laser" and "destroy object" things worked (at least based on my experience using the Games Factory) is that they were considered standard things to do by the editor. All the assets that came with it had their own death/explosion animation and I think you may even have been able to assign "shoot" from the right-click context menu (or I could just be misremembering that). Doing anything other than shoot/destroy/change variable essentially required you to program it yourself (admittedly in words).

Also, all the programming logic for a given level was just stored as one big list and I'm not sure how well something like this would translate to a more structured context, e.g if you implemented something similar in Unity where would the logic go? You could probably make something like that work, but it's not quite as good as it sounds (though I haven't used it for years so I may not be remembering it very well).

I tried the lite version of Games Factory (which is coming from the Multimedia Fusion company). I like it, but I don't think that it speaks to the creative side of the brain... Instead of using a programming language, you use a GUI, but you are still trying to solve logical problems. You are still coding. I would think that there is not much difference to programming.

Actually, I have the feeling that I can be more creative when programming. Also when programming there are way more possibilities for unknown things happening.

Also, MMF and GF are great for platforming games (like the stuff what Nifflas did). I wonder if there are too many restrictions for doing other stuff...

I might need more practice to get fast with it... reminds me a lot of Wario DIY for the NDS. You could make very easy micro games. It got tedious when you wanted to implement a certain feature, but the components, the software bricks were not flexible enough, so you had to use a lot of invisible helping objects to get your logic done.

Ian Bogost's new project Game-O-Matic (http://game-o-matic.com/) is a very interesting take on this problem. You make a concept map of things and verbs that connect them, and it randomly generates little games based on them.

You can see a video here:
http://www.youtube.com/watch?v=Y7KacgqA8cI

I finally got around to skimming through Bret Victor's Magic Ink (http://worrydream.com/MagicInk/), and it actually talks about the idea of "expressing general logic through specific examples" that I'd mentioned earlier (http://evolutionlive.blogspot.com/2010/09/how-artists-want-to-make-games.html).

There's even a name for it - it's called programming by demonstration (http://en.wikipedia.org/wiki/Programming_by_demonstration). I'm imagining demonstration mostly by diagrams, but there could be other ways to do it.

We're getting closer... :)

It's well worth a thorough read. :)

Make sure to hit up Ladder of Abstraction if you get the chance. His famous video is simply logical conclusions of the principles he lays out in that article.

http://www.unity3d.com/ninjacamp/projects/10180 (http://www.unity3d.com/ninjacamp/projects/10180)

It would appear that someone at Unity is interested in Bret Victor, even if they did misspell his name. Although it dosen't seem that there's going to be anything close to his ideas in Unity any time soon, at least not officially.

Bret Victor has a new article up on his site, and it is so awesome:

Learnable Programming (http://worrydream.com/LearnableProgramming/)

YES!! ;D

A lot of the stuff that we want, he talks about being a necessity. Now how do we make this actually happen... :)

The solutions he proposes are still code-centric.

Making feedback instant & easier to "read" is great ( for programmers ), but you're still programming / writing syntax instead of working directly on the end-result.

(http://i.imgur.com/wKueg.png)

Why type shape("rect",80,20,140,60); if you could draw a rectangle on the canvas instead.

(http://i.imgur.com/ggZvO.png)

People seem to have forgotten that "code" is ( binary ) data just like anything else.

Some programmers -most?- are very hostile towards visual interfaces to programming. They keep finding better ways to make drawings by pressing the cursor keys on the keyboard, hoping that at some point the artists will admit that cursor keys are so much more convenient than brushes and pencils.

(while in fact it always feels like they are -consciously or not- excluding artists from the domain they believe is their own)

I think that programming is fundamentally different from something like painting or music. I do both, and I feel a definite shift in my state-of-mind when switching between the artistic and engineering aspects of game-making. This is, however, during the development of a game engine. Once the game engine is completed, I'm going to try to make it so I can manipulate things using the mouse in real-time. From what I've tried of visual programming (A little bit of Puredata), it works better with simple relations, and it's very good for experimenting. I agree with what people have said earlier about real-time programming being a lot better than the process of changing something, compiling, and seeing what it does, like a film photographer. I think visual interfaces aren't more popular because they're pretty domain-specific -- good for visual or musical programs, but not for much else. I don't have much experience, but that's my feeling. I think a visual interface might be cumbersome for implementing something like AI, and probably wouldn't work too well for text-based games. As for the "pressing cursor keys to make paintings" idea, I'm not sure if you're referring to the procedural generation of art or something else. I think most big game studios have ways that artists can arrange things to their liking after the infrastructure is completed.

Edit:
By the way, there are a lot of neat real-time programming systems out there, many of them designed for audio/visual live-coding (programming as a performance art, a pretty new thing). The one I'm most familiar with is Supercollider, which I'm planning on integrating with a game at some point. Toplap.org has a lot of info on these.

I have a lot of experience with visual programming. The Endless Forest and The Path are programmed visually. I also created an AI system visually -Drama Princess. Our newest Bientôt l'été is also programmed visually. The prejudices against it are wrong. Visual programming can be more powerful and more versatile, at least for some people. The only thing it is not is easy and quick. It has that in common with code-based programming. Programming is always complex. That's kind of the beauty of it.

Hi all (first post), I have also much experience with Quest3d finding its visual programming a really powerful method, however it is actually abandoned by act3d:
http://lumionautics.com/interview-with-ferry-marcellis/ (http://lumionautics.com/interview-with-ferry-marcellis/)

Michael, wondering what made you stay away from Q3d (you were right). Maybe the mono-platform?
What would be to your eyes perfect-visual programming? Something in between Q3d and Unity?

Yes, the main reason why we use Unity is its multi-platform capacity. I still consider Quest3D to be a more powerful programming system, for people like me. Thanks to the visual programming add-on Antares Universe, I can finally tackle more sophisticated projects in Unity. But this add-on is limited by Unity's (almost) absent real-time interpretation (everything needs to be compiled before it can be run and that seriously limits the directness required for a lot of artistic creativity).

I have actually designed a real-time programming concept (when discussing such a feature in a Unity beta group). I should find a programmer and make a working mock-up of this concept to see if it would actually work. It was specifically designed with Unity in mind, though. So it's probably not as ideal as it could be. Though I think it would make programming a lot more fluent -and a lot more fun- for many people.

I'd be very interested in hearing your idea. Would you be willing to share it on this forum. :)

Same here.

Here's the sketches I made. Hope they are comprehensible:
(remember this was conceived with Unity in mind, as an alternative to scripting)

http://tale-of-tales.com/tales/LogicGraph/ideas01.html
Overview of the concept.

http://tale-of-tales.com/tales/LogicGraph/ideas02-opendoor.html
How a typical game interaction would be graphed, described at the top of the page.

http://tale-of-tales.com/tales/LogicGraph/ideas03-NinjaStar.html
A small game suggested as a challenge for this system, described at the bottom.

Nice & clean concept .. one that could easily translate into the viewport as well. I took the liberation to make the "Key" a child of the Player ( as boolean variable ) ;)

(http://i.imgur.com/cEyKc.jpg)

- Once you select object(s) in the viewport the child nodes are shown.
- When dragging nodes outside of their parent, their 3D position is calculated relative to the projection plane.
- When you roll-over the "Door.Open" animation node, you'd instantly see the animation being previewed in the scene
- Etc :)

Very nice!
I was definitely considering a one-to-one connection with the structure of the scene hierarchy in Unity but had not thought about representing the logic in the viewport. This is very exciting!

We might still need a more abstracted view, for logic that is not directly connected to 3D objects.
Unless perhaps we show abstract "objects" in the viewport as well. That could get messy. But might be easier to navigate than the current hierarchy + inspectors.

Maybe the logic graph can even be three-dimensional!

What if we consider comic strip language? Logic connected to objects could be represented as thought bubbles…

Could you give a example of a script that doesn't belong to any object? :)

Well, of course, in Unity, every bit of logic is connected to an object in the hierarchy. But sometimes there's a lot of logic. And sometimes the object is just an empty object (and thus invisible). Logic that is primarily calculations, arrays, loops and conditions. An A.I. system would be an extreme example, or any sort of Finite State Machine.

But as I mentioned, perhaps elements of abstract logic could also be presented in the 3D space as objects that are not rendered in the game but only visible in the scene viewport (or you could simply put them on a layer that the game camera doesn't render). It might be very intuitive actually to manipulate logical "objects" in a similar way as actual 3D objects.

An array could be a stack of cubes with physics properties!  :)

AI drives characters, and there's allot you can do to visualize the underlying system through that character.


Same thing, a array serves a purpose to something. It's all about the context.

Just like the bits 1000001011011010110010101101110 can be the word "Amen", the value 14.84 or the following color depending on their context.

(http://i.imgur.com/lgeET.png)

All of which you'd visualize in a completely different way ( even though it's the same data ).

To prove his point about a lot of logic, here's two examples of some level design / scripting work I've done using visual logic nodes in the Marmoset engine. The first image shows logic for a battle with advancing lines of infantry, firing cannons, etc.
(http://www.torncanvas.com/content/10.videogames/8.darkest-of-days/02.jpg)
(http://www.torncanvas.com/content/10.videogames/8.darkest-of-days/01.jpg)

Wow, that is pretty awesome, thanks for putting together those example diagrams for us! ;D

I like the bright colors - I can almost see the nice bouncy easing curves on those nodes shrinking and growing and sliding and locking together. Do the colors have significance in your diagrams? (representing data types, maybe?)

I'm still digesting how it all would work, but this is already inspiring. :) I wonder how we could add visualization of data and dynamics (as Bret Victor suggests in Learnable Programming) in addition to your visualization of structure...

Not necessarily. Perhaps the array is used in a general sorting algorithm that should be applicable in any context, for any purpose. Sometimes you're going to need abstraction.

That's why I mentioned how bits are interpreted. Even though a logic / memory block is used for various purposes, you want to visualize the data as a color when it's used as a color, and as a string when it's used as text. The same can be applied to whatever you're actually sorting using the algorithm.

What about the debugging process? Looks like it would be hard.
May this visual programing run in a layer over everything else so you can introduce break-points or similar?

Could you explain a little bit what this image represents? I'm not familiar with this engine.

I think I only use black (and grey) consistently in the diagrams.
The idea of coloring nodes by data type is nice but often gets too limited. Maybe data types should be represented as different shapes. Colors are very handy to make connections between elements. That's how I've used them. One of the big problems I have when programming is keeping track of how everything is connected with each other. This is the main reason why I feel that the entire game should be represented in a single graph (as by cutting things up, one quickly loses a sense of how they fit together and which piece of logic is calling which other piece of logic, or setting its parameters, etc).

One thing I have learned is that with visual programming, the number of bugs is drastically reduced. If only because it's impossible to make spelling or syntax errors.

Mostly, in visual logic, when the game is running, the active parts of the graph are highlighted. That helps a lot when debugging. Also, since this needs to run in real-time, you can disconnect parts of the graph while the game is running and see if an error still occurs, etc. And since every parameter can be represented as a visible node, you can see its value update in real-time, without needing to add Debug.log logic.

The black boxes are invisible walls, the yellow ones are triggers that activate, with a customizable filter for any character, just the player, etc. Some of the brighter small squares are visual effects and triggering animations if I remember right. The darker magenta circles are logic nodes, so you can link that to a trigger that passes messages to other objects. I think an example would be to trigger a line of men to advance when a certain number of enemy men die; something like that.

Fascinating. So Marmoset is a game authoring tool?
Does anyone know any games created with it?

It's a custom C++ game engine developed by 8monkey Labs. That tool is the level design/scripting tool called Habitat; when I used it, it was fairly limited in features, being supported by our 2-person programming team and pretty focused on the game we were working on at the time, Darkest of Days. I think that's the only shipped title that's been developed with it. The model viewer Toolbag (http://www.marmoset.co/) is used by the public at large, mostly by AAA game artists in the Polycount community.

I just read the old book Mindstorms (http://www.amazon.com/Mindstorms-Children-Computers-Powerful-Ideas/dp/0465046746), by Seymour Papert, after Bret Victor recommended it so highly in his article Learnable Programming.

It's a good one - got me inspired about education again. :P Still trying to figure out how to actually apply these ideas and the example of Logo to a game-making tool...

I've just discovered another awesome Bret Victor project, this time it's the most artist-friendly visual programming system I've ever seen, Substroke (http://worrydream.com/#!/substroke).

(http://worrydream.com/substroke/overview_directobject.png)

It's mostly an academic thing, though I think he's made some sort of prototype of it. As it stands, it's still pretty arcane, but it's a big step in the right direction.

I really think "programming via visual storyboards a la comics" is the way to go. Like this (http://worrydream.com/MagicInk/#p272). Maybe supplemented with a natural-ish language "script" that it translates to, where you can edit at either level.

I finally understand how functional programming can work for real-time interactive graphical applications! :D There's a new functional language called Elm (http://elm-lang.org/) that compiles to HTML, CSS, and JavaScript for making websites and games.

The way it works is called functional reactive programming (http://elm-lang.org/learn/What-is-FRP.elm), which basically means it recalculates the state of the virtual world whenever input values (keyboard, mouse, timers) change, and the graphical representation is calculated from the state of the virtual world so that updates too. All the code for the game basically describes the way this data flows and is transformed (through lots and lots of nested functions). As a result, I think it could lend itself very naturally to a visual representation. It's basically just a big tree, a network of data flowing and combining, and if you see this structure you see the entire program.

This could be great for both R-mode readability and building with functional (no pun intended) pieces in real time.

You can see a number of simple examples (http://elm-lang.org/examples/Intermediate.elm), including a detailed explanation of the code behind Pong (http://elm-lang.org/blog/games-in-elm/part-0/Making-Pong.html).

Not exactly the prettiest code at first glance (probably more confusing than the kind of programming we're used to). I think the real potential here is in making it visual. We all know how Scratch looks, trying to make normal code visual. I think functional programming would actually look better in visual form than in text.

And in the long run, I bet functional programming would be much more amenable to programming by demonstration (http://worrydream.com/MagicInk/#p272) - that is, expressing general logic through specific examples.

I'm a bit late with this, but... behold, Bret Victor does it again!

Stop Drawing Dead Fish (http://vimeo.com/64895205)

That's the kind of programming-by-drawing that I've been trying to figure out how to do for a long time now. He's actually got a simple system that totally works. Now I just have to figure out how to apply it to games instead of just procedural animations...

I really hope there's some people implementing his research in the field of videogames.

Also, it's important to note that he has a great system for programmatic animation, but he hasn't really figured out logic yet. Right now his solution is choosing pre-made actions with sentence syntax. It'd be great to see someone taking his approach and going after logic.

Indeed. Bret Victor always strikes me as someone with good intentions but too little imagination (or too shallow awareness of what makes visually inclined creators most productive; let alone of what they might want or be able to make).

Oh. Well I find him to be incredibly imaginative. ;D But he's just one guy, and moving things around seems like a good first step before logic. I just hope there are other people out there building off his ideas. IMO they're too good to just leave alone.

Yes, he's just one guy, and these are all baby steps. But they're in the right direction. It is amazing how few people are running with this. I'd really like Linden Lab to dive into this stuff, but I'm not sure I can make that happen...

This is a really long thread, I don't know if I can possibly read it, but I was just about to make a thread about this subject, so I will leave a link here afterward.

I don't think though that you really want visual programming. It sounds like a good alternative, but it's going to get you to a place that is a jumble of knots impossible to untangle. I think you want literary programming...

EDITED: http://notgames.org/forum/index.php?topic=853.0