SCNTechnique in SceneKit
New in ios8 is the SCNTechnique. It’s a way to get your models to pass through a custom shader program. This is an added layer on top of an objects filters. For instance, you’re able to apply a Gaussian blur to a single node in a SceneKit scene, then pass the scene to a fragment shader to change the color of the scene.
新的ios8 SCNTechnique技术。这是一种让你的模型通过一个自定义着色器程序。这是一个添加过滤器层上的一个对象。例如,你能够应用高斯模糊SceneKit现场单个节点,然后通过场景片段着色器改变现场的颜色。
Getting started I’m setting up a new ios game.
1、开始我设置一个新的ios游戏。
[img][/img]
This will create the scene view that SceneKit uses to render it’s 3d objects to. I’ve named the project SCNTechnique, this will make it obvious what this project is being used for.
2、这将创建SceneKit使用的场景视图呈现它的3 d对象。我将该项目命名为“SCNTechnique,这将使它明显被用于这一项目。
I’m also enabling the local source control, I think git is required for this to work. If you don’t have this then I suggest you use it. If you break something it’s useful to know what you changed.
3..
In the project we’ll want to set up a new dictionary that’s used to reference and define data going to and from the SCNTechnique rendering pass. I’m adding an iOS property list. Selecting add new file to the project to open this choose a template for your new file dialog:
4、项目中我们想要建立一个新的字典,用来引用和定义数据和从SCNTechnique呈现通过。我添加一个iOS属性列表。选择将新文件添加到项目中打开这个为你的新文件对话框:选择一个模板
Then I’m naming it firstPass, it’ll add the plist file extension for me. A property list is an XML file which Xcode allows us to edit with a more friendly interface.
5、然后我给它命名firstPass,它会增加我的plist文件扩展名。一个Xcode属性列表是一个XML文件允许我们编辑更友好的界面。
The file appears with Root as the only section of data. We need to populate this with a few things to turn it into something that can be used by SCNTechnique. There are three entries [sequence, passes, symbols] where sequence is an Array and passes and symbols are both Dictionaries. These are added with the plus icon, and the type is changed with a popup menu under the type column.
6、文件只出现与根部分的数据。我们需要填充这几件事,把它变成SCNTechnique可以使用的东西。有三个条目(序列,通过符号),序列是一个数组,通过和符号都是字典。这些添加的加号图标,弹出菜单的类型是改变类型列。
sequence defines what pass is rendered when. To be honest, I think this means that you can define multiple passes in a single SCNTechnique, but so far this seems to be quite buggy and I can’t seem to get more than one pass working. Passes defines what parameters are passed to and from the shader programs. We’ll get to shader programs in a moment. symbols defines what name and type the data is passed from your code into the shaders.
7、序列定义了通过时呈现。老实说,我认为,这意味着您可以定义多个传入一个SCNTechnique,但到目前为止,这似乎是相当车,我似乎无法得到不止一个传递工作。通过定义了参数传递的着色器程序。我们会着色器程序。符号定义了名称和类型的数据从您的代码传递到着色器。
So shader programs, I couldn’t find any appropriate templates to start with for these, so I ended up picking iOS/Other/Empty template. Which isn’t much of a template… But it is necessary to ensure that the project is aware of your new file. I added two files, a firstProgram.vsh and a firstProgram.fsh.
8、所以着色器程序,我找不到任何合适的模板,从这些开始,所以我最终选择iOS /其他/空模板。这并不是很大的模板…但它是必要的,以确保该项目是意识到自己的新文件。我添加了两个文件,一个firstProgram。太小,firstProgram.fsh。
These live in the project like so.
What is the fsh and vsh for? the fsh means fragment shader and the vsh means vertex shader. Basically these are common OpenGL type shader programs. To make the connection between these shader programs and SCNTechnique in your SceneKit scene you’ll need to add them to the firstPass.plist so back to defining the technique’s property list.
9、fsh和vsh是什么?fsh意味着片段着色器和太小意味着顶点着色器。基本上这些都是常见的OpenGL渲染程序类型。这些着色器程序之间的联系和SCNTechnique SceneKit场景你需要将它们添加到firstPass。plist所以回定义技术的属性列表。
In the Sequence array we need to name a rendering pass in the passes list. This seems to be the entry point for the technique. I’m naming it renderingPass.
10、序列数组中我们需要呈现通过通过列表中的名称。这似乎是入口点的技术。我命名它renderingPass。
This tells the Technique where to start. The item 0 of the sequence needs to point to an entry in the passes item. Then in passes we are require to have 4 entries.
11、这告诉技术从哪里开始。0项的序列需要指向一个条目的条目。然后在通过我们需要有4个条目。
draw program inputs and outputs, draw is a string program is another string and inputs and outputs are both dictionaries.
12、画程序输入和输出,平局是一个字符串的程序是另一个字符串,输入和输出都是字典。
draw needs to be set to DRAW_QUAD which is used to draw data piped into the shader program, we also have DRAW_NODE or DRAW_SCENE. Each is used for specific reasons, but for now we’ll use DRAW_QUAD. The program refers to the .vsh and .fsh files we created and added to the project a moment ago. (note: the case and underscore are important) Don’t add the .vsh or .fsh to the program, Technique knows what to do with the name. I’ll always look for both files, if one or the other are missing the Technique will fail and your app will stop when looking for the program.
13、将需要设置DRAW_QUAD用于将数据输送到着色器程序,我们也有DRAW_NODE或DRAW_SCENE。每个用于特定的原因,但现在我们将使用DRAW_QUAD。程序指的是。太小。fsh文件创建和添加到项目中。(注:和下划线)很重要不要添加。太小。fsh的项目,技术知道如何处理这个名字。我总是会找两个文件,如果一个或另一个缺少技术会失败,应用程序会停止当寻找项目。
It seems rather necessary that one of the inputs is COLOR which goes into a colorSampler. COLOR is an openGL data source and colorSampler will be available in the shader program with data coming from COLOR. the a_position is a variable which points to a_position-symbol. We will need to add this to the symbols section of the technique’s plist. So we’ll add that in next.
14、似乎而必要的输入是进了colorSampler颜色。颜色是一个openGL数据源和colorSampler将着色器程序中可用的数据来自颜色。a_position是一个变量,指向a_position-symbol。我们将需要添加这个符号部分技术的plist。所以接下来我们将添加进去。
In the a_position we add a_position-symbol and in symbols create a_position-symbol as a dictionary and add semantic vertex string to create a link between your SCNTechnique and the program. I’m not exactly sure why we don’t need to do this for colorSampler, but that’s how it works.
now we should add our code to both the .vsh and .fsh to make this work. In the firstProgram.fsh I’ve added the following code:
uniform sampler2D colorSampler;
varying vec2 uv;
void main()
{
vec4 color = texture2D(colorSampler, uv);
gl_FragColor = color;
}
This isn’t objective-c it’s c, and the colorSampler here is being piped in from the plist which SCNTechnique is using to tunnel data through. gl_fragColor is the final output to the fragment shader.
In the vsh we need to add in the following code:
attribute vec4 a_position;
varying vec2 uv;
void main(void)
{
uv = (a_position.xy + 1.0) * 0.5;
gl_Position = a_position;
}
The gl_Position is the final output for the vertices in the vertex shader. With these two parts of the firstProgram in place and the firstPass.plist setup we can now use it in the GameViewController.m
...
// configure the view
scnView.backgroundColor = [UIColor blackColor];
NSURL *url;
url = [[NSBundle mainBundle] URLForResource:@"firstPass" withExtension:@"plist"];
SCNTechnique *firstTechnique = [SCNTechnique techniqueWithDictionary:[NSDictionary dictionaryWithContentsOfURL:url]];
scnView.technique = firstTechnique;
...
In the template code we follow after the scnView is setup in the scene. Then we create a NSURL *url; to fill with url = [[NSBundle mainBundoe] URLForResource:@”firstPass” withExtension:@”plist”]; this makes a url that points to the firstPass.plist that we created with all of the data used to connect our shader program with the SCNTechnique. The we use SCNTechnique *firstTechnique to load in that dictionary. Finally we assign scnView.technique to the SCNTechnique we created from the plist.
IMG_0104
Horray, we’re drawing using a SCNTechnique, it doesn’t look like anything interesting is happening, but it’s really going through our shader program and not the usual scene.view. We can prove this by editing the shader program. We can draw the technique as red by changing the fsh to the following code.
uniform sampler2D colorSampler;
varying vec2 uv;
void main()
{
gl_FragColor = vec4(1.0, 0.0, 0.0, 0.0);
}
This sets the gl_fragColor to red. And now the iPad draws the image as all red. I’m still working on more interesting things to do with all of this but if you’re familiar with openGL then you’ll be able to do quite a lot. I’m not that familiar with openGL myself, so I’m learning what I can while I can. Hope this tutorial helps.
Read More
July 25, 2014
SceneKit shaderModifiers
So I started with the basic GameKit, this gives you a colorful jet object in the scene.
By default the code in the GameViewController.m looks like the following:
55
- (void)viewDidLoad
{
[super viewDidLoad];
// create a new scene
SCNScene *scene = [SCNScene sceneNamed:@"art.scnassets/ship.dae"];
// create and add a camera to the scene
SCNNode *cameraNode = [SCNNode node];
cameraNode.camera = [SCNCamera camera];
[scene.rootNode addChildNode:cameraNode];
// place the camera
cameraNode.position = SCNVector3Make(0, 0, 15);
// create and add a light to the scene
SCNNode *lightNode = [SCNNode node];
lightNode.light = [SCNLight light];
lightNode.light.type = SCNLightTypeOmni;
lightNode.position = SCNVector3Make(0, 10, 10);
[scene.rootNode addChildNode:lightNode];
// create and add an ambient light to the scene
SCNNode *ambientLightNode = [SCNNode node];
ambientLightNode.light = [SCNLight light];
ambientLightNode.light.type = SCNLightTypeAmbient;
ambientLightNode.light.color = [UIColor darkGrayColor];
[scene.rootNode addChildNode:ambientLightNode];
// retrieve the ship node
SCNNode *ship = [scene.rootNode childNodeWithName:@"ship" recursively:YES];
[ship runAction:[SCNAction repeatActionForever:[SCNAction rotateByX:0 y:0.2 z:0 duration:1]]];
// retrieve the SCNView
SCNView *scnView = (SCNView *)self.view;
// set the scene to the view
scnView.scene = scene;
// allows the user to manipulate the camera
scnView.allowsCameraControl = YES;
// show statistics such as fps and timing information
scnView.showsStatistics = YES;
// configure the view
scnView.backgroundColor = [UIColor blackColor];
// add a tap gesture recognizer
UITapGestureRecognizer *tapGesture = [[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(handleTap:)];
NSMutableArray *gestureRecognizers = [NSMutableArray array];
[gestureRecognizers addObject:tapGesture];
[gestureRecognizers addObjectsFromArray:scnView.gestureRecognizers];
scnView.gestureRecognizers = gestureRecognizers;
}
There’s not much going on other than adding models, lights and cameras to the scene. I wanted to do some learning up on shaderModifiers so I started by getting the mesh in the ship.dae.
To do that I need to get to the material on the ship.
1
ship.geometry.firstMaterial
The above shows up with nothing, the ship included in the scene has no geometry in the ship node. This took a few minutes of fiddling before I found that the ship has a shipMesh child object.
Screen Shot 2014-07-25 at 6.48.44 PMThe child is called shipMesh, and that’s the node that would have a firstMaterial. To get to that I used
1
SCNNode *shipMesh = [ship childNodeWithName:@"shipMesh" recursively:YES];
The line assigns the actual mesh I need to get to the *shipMesh SCNNode object. now I can use the following:
1
SCNMaterial *mat = shipMesh.geometry.firstMaterial;
This assigns the shipMesh’s material to the SCNMaterial *mat so I can start adding shaderModifiers to the object in the scene.
Starting with the basics I tried out:
1
mat.shaderModifiers = @{SCNShaderModifierEntryPointFragment : @"_output.color.rgb = vec3(0.5);"};
this turned the ship to a grey color. on the iPad:
Screen Shot 2014-07-25 at 6.55.28 PMSo this was expected, vec3(0.5) tells GLSL to make a rgb color of r 0.5, g 0.5. b 0.5, it’s a handy short-hand for color making. Another thing I tried was the following.
1
mat.shaderModifiers = @{SCNShaderModifierEntryPointFragment : @"_output.color = vec4(0.5);"};
This wasn’t assigning color.rgb but color, which expects a vec4, and I got the same result; a grey jet appears on the iPad. Just to be thorough, I tried assigning an interesting value to the color.
1
mat.shaderModifiers = @{SCNShaderModifierEntryPointFragment : @"_output.color = vec4(0.5, 0.1, 0.2, 0.3);"};
The above turned into a reddish jet.
Screen Shot 2014-07-25 at 6.58.46 PM
So this again was expected. I took some code from the WWDC presentation and found some code to try out on the jet.
mat.shaderModifiers = @{SCNShaderModifierEntryPointGeometry : @"// Waves Modifier\n"
"float Amplitude = 1.2;\n"
"float Frequency = 5.0;\n"
"vec2 nrm = _geometry.position.xz;\n"
"float len = length(nrm)+0.0001; // for robustness\n"
"nrm /= len;\n"
"float a = len + Amplitude*sin(Frequency * _geometry.position.z + u_time * 10.0);\n"
"_geometry.position.xz = nrm * a;\n"};
The above made the jet all wobbly.
Screen Shot 2014-07-25 at 7.00.46 PMAll of the verts were going off of a sin(u_time) which is a value in the GLSL world that all shaders have access to. Cool stuff so far. After this I found some interesting car paint shader from the WWDC presentation and tweaked it to get the following result.
mat.shaderModifiers = @{SCNShaderModifierEntryPointSurface : @"\n"
"float flakeSize = sin(u_time * 0.2);\n"
"float flakeIntensity = 0.7;\n"
"vec3 paintColor0 = vec3(0.9, 0.4, 0.3);\n"
"vec3 paintColor1 = vec3(0.9, 0.75, 0.2);\n"
"vec3 flakeColor = vec3(flakeIntensity, flakeIntensity, flakeIntensity);\n"
"vec3 rnd = texture2D(u_diffuseTexture, _surface.diffuseTexcoord * vec2(1.0) * sin(u_time*0.1) ).rgb;\n"
"vec3 nrm1 = normalize(0.05 * rnd + 0.95 * _surface.normal);\n"
"vec3 nrm2 = normalize(0.3 * rnd + 0.4 * _surface.normal);\n"
"float fresnel1 = clamp(dot(nrm1, _surface.view), 0.0, 1.0);\n"
"float fresnel2 = clamp(dot(nrm2, _surface.view), 0.0, 1.0);\n"
"vec3 col = mix(paintColor0, paintColor1, fresnel1);\n"
"col += pow(fresnel2, 106.0) * flakeColor;\n"
"_surface.normal = nrm1;\n"
"_surface.diffuse = vec4(col.r,col.b,col.g, 1.0);\n"
"_surface.emission = (_surface.reflective * _surface.reflective) * 2.0;\n"
"_surface.reflective = vec4(0.0);\n"};
Screen Shot 2014-07-25 at 7.04.07 PMStrangely enough I didn’t delete the code that made the jet wobbly, so I’m guessing something was overwritten. The shaderModifiers is being assigned one value when we use = to assign the value an @{“”}; object. So a bit of looking at the shaderModifiers object in the materials and I see that it’s an NSDictionary. The NSMutableDictionary and NSDictionary are supposedly compatible, so I make a new NSMutableDictionary with the following:
1
NSMutableDictionary *shaders = [[NSMutableDictionary alloc]init];
I’ll use that and assign multiple keys with code to the *shaders dictionary.
this looks like the following:
NSMutableDictionary *shaders = [[NSMutableDictionary alloc]init];
shaders[SCNShaderModifierEntryPointGeometry] =
@"// Waves Modifier\n"
"float Amplitude = 1.2;\n"
"float Frequency = 5.0;\n"
"vec2 nrm = _geometry.position.xz;\n"
"float len = length(nrm)+0.0001; // for robustness\n"
"nrm /= len;\n"
"float a = len + Amplitude*sin(Frequency * _geometry.position.z + u_time * 10.0);\n"
"_geometry.position.xz = nrm * a;\n";
shaders[SCNShaderModifierEntryPointSurface] =
@"float flakeSize = sin(u_time * 0.2);\n"
"float flakeIntensity = 0.7;\n"
"vec3 paintColor0 = vec3(0.9, 0.4, 0.3);\n"
"vec3 paintColor1 = vec3(0.9, 0.75, 0.2);\n"
"vec3 flakeColor = vec3(flakeIntensity, flakeIntensity, flakeIntensity);\n"
"vec3 rnd = texture2D(u_diffuseTexture, _surface.diffuseTexcoord * vec2(1.0) * sin(u_time*0.1) ).rgb;\n"
"vec3 nrm1 = normalize(0.05 * rnd + 0.95 * _surface.normal);\n"
"vec3 nrm2 = normalize(0.3 * rnd + 0.4 * _surface.normal);\n"
"float fresnel1 = clamp(dot(nrm1, _surface.view), 0.0, 1.0);\n"
"float fresnel2 = clamp(dot(nrm2, _surface.view), 0.0, 1.0);\n"
"vec3 col = mix(paintColor0, paintColor1, fresnel1);\n"
"col += pow(fresnel2, 106.0) * flakeColor;\n"
"_surface.normal = nrm1;\n"
"_surface.diffuse = vec4(col.r,col.b,col.g, 1.0);\n"
"_surface.emission = (_surface.reflective * _surface.reflective) * 2.0;\n"
"_surface.reflective = vec4(0.0);\n";
mat.shaderModifiers = shaders;
The different SCNShaderModifierEntryPointGeometry and PointSurfaces are key values with objects assigned to them. At the end we assign mat.shaderModifiers = shaders to assign both of these objects in the NSMutableDictionary to the material. So now the final result looks like the following:
Screen Shot 2014-07-25 at 7.09.39 PM
A wobbly pink jet. The significance of all of this is that to apply multiple shaders to the object you need to create a dictionary first, and use the different entrypoint constant values as keys. Hope this all made sense.
Read More
June 18, 2014
Looking for some Unreal Stuff?
http://okita.com/alex/unreal-game-development-archive/
The archives and what’s been going on can be found on the link above. Updates and adventures will be forthcoming as well as a few project announcements.
IMG_20140610_083914Making the most of a small workshop means approaching a critical density of stuff. I need to organize this into something more sensible, and some cool gizmos will be coming to this site soon. Check back often!