Hi,

Short intro -> Im a CS student, knowledge of basic OpenGL and java... and in need of some lovely help...im a novice with android so sorry if im making stupid mistakes..

im currently making a free game for one of my projects. For test purposes im outputting aload of 3d triangles. i need to select certain objects and move them in a certain direction. After researching i found it can be quite tricky on android devices to implement a picking technique which selects objects based on where you touch the screen and read the framebuffer.

All i need to know is to know which object(square/triangle etc) im touching. To do this i render all the objects, when a touch event occurs it clears all past rendering and re renders the scene with each objects own special 'picking' method which draws the objects with a unique colour. i use glReadPixels() to find out where im touching and it reads the colour been displayed at that point. To make sure im not reading incorrect colours i have disabled all effects such as textures/fog etc.

The problem im encountering is that glReadPixels always reads 0 0 0 0 which is black???... even when selecting something rendered... im not entirely sure on how to fix this problem and would like some help if possible or perhaps a hint in the right direction thanks

The following code examples are a cut down version of what im currently using and some DDM output:




Inside the Triangle object class:





any help would be greatly appreciated! sorry of the code looks choppy..

After looking for a solution this morning i believe glReadPixels is working... what lies the problem is that i need permission to access the framebuffer to gain the information?? the way to do this changed when android went from 1.0 to 1.5.. i believe... hence until i do this ill get 0 0 0 0 results? im still searching for a solution and hopefully ill get one soon. as soon as i get this working ill probably write a tutorial as this is driving me crazy so no idea what its doing to other novices...


so far ive added the following permissions to the manifiest files:


and this managed to help me so far point in the right direction:

http://osdir.com/ml/android-ndk/2009-12/msg00171.html

The post was very useful...


I am getting 0,0,0,0 when using glReadPixels. I think its the permission issue.

When I try to get the permissions it fails telling that the protection level for this is 2.

As I understand these permissions are not for the 3rd Party developers. I am trying it only for learning purposes and not releasing this.

Any workaround will be useful. If possible plz tell me in detailed steps.

Thanks.

Hi,

Sorry to see your having the same problem... eventually I gave up and implemented boarders around displayed objects. It’s a very tedious and no way near as accurate but works for what I wanted to achieve. For a simple puzzle game it works ok as I always know what position my camera and not too bad. However for a more complex 3d game or app I wouldn’t recommend this approach.

For another solution you can try ray tracing. Based on the camera position and where the user touches you can create a "ray" to x distance. You then can increase this ray x amount and if it touches an object within x-max then you have a collision (aka every time you increase the ray check for any collisions). If so win otherwise ignore.

p.s. very late im tired few errors above aha

Hi folks,

i am currently struggling with the same problems. I need a way of picking objects on my screen and looked into glReadPixel with coloring.. no success until now.

I will definetly let you know if and how it worked for me. But until then i appreciate any help from someone else that got it working already!

Greets,

atla

I nailed it down more or less, i get different colors from my 3x3 quads with different red tones... but due to the conversion to Float and back to int i don't get the correct values.

My Example:

to test i took bigger distances between the colorcodes for my first try.

bla.pickId = i * 20 (e.g. 60 for third tile)

i render using this:

gl.glColor4f(pickId/255.0f, 0.0f, 0.0f, 1.0f);

this.renderer.draw(gl);

i guess i already loose precision on that conversion as the OpenGL ES GL0 implementation does not offer the glColor4ub function i would use on standard GL... therefore i need to use the color4f function here.

when i read my Pixels i read it like the example above looking for bytes

byte[] c = new byte[4];
ByteBuffer pixels = ByteBuffer.wrap(c);

gl.glReadPixels(pickX, height-pickY, 1, 1, GL10.GL_RGBA, GL10.GL_UNSIGNED_BYTE, pixels);

int r = c[0];
int g = c[1];
int b = c[2];
//int a = c[3];

if (r < 0)
r += 256;
if (g < 0)
g += 256;
if (b < 0)
b += 256;

Log.v("TEST", "bla r " + r + " g " + g + " b " + b );

My problem is i do not get the correct values as put in pickId but get values like 99, 123, 82 instead of 100, 120, 80 ...
i could work around this and just round up to the nearest power of 20 but i would rather like to be able to pick way more objects

maybe someone has an idea for that?

Atla, have you found a solution to that problem?

By the way, to (possibly) avoid rounding errors I'm using glColor4x like this:
glColor4x(red*257, green*257, blue*257, alpha*257);
Where red/green/blue/alpha = [0..255]

Hi,
i am using the color picking method in my app and picking works fine!

I have no rounding errors with:

drawing:

Code: Select all

int redDraw = value; // < 256
gl.glColor4f(redDraw, 0, 0, 1);


searching object after picking:

Code: Select all

// i get a byteBuffer with RGBA
ByteBuffer buffer = ...
int red = buffer.get(0);
if(red < 0)
{
   red += 256;
}


BUT i have the problem, that all my coloured objects are coloured on the screen.
I think the back buffer is not cleared before redrawing all objects with textures.

What is the right way to clear the back buffer in OpenGL ES 1.0?

Hier is my Code:

In onTouchEvent-method (getting the index of the right object)

Code: Select all

      int iIndex = -1;
      int iRed;
      
      m_PickingPixelBuffer = ByteBuffer.allocateDirect(4);
      m_PickingPixelBuffer.order(ByteOrder.nativeOrder());

      // setting globals for Renderer
      m_bPickingEnabled = true;
      m_iPickingX = _iX;
      m_iPickingY = _iY;
      requestRender();

      // wait for picking process (renderer)
      while(m_bPickingEnabled);

      iRed = m_PickingPixelBuffer.get(0);
      if(iRed < 0)
      {
         iRed += 256;
      }
      
      for (int i = 0; i < numOfObjects; i++)
      {
         if (m_Modell.getCover(i).getColorRed() == iRed)
         {
            iIndex = i;
            break;
         }
      }


Drawing

Code: Select all

public void onDrawFrame(GL10 gl)
   {
      // clear Buffer
      gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);

      if (isPickingEnabled())
      {
         // draw colored objects  and picking pixel color
         pickingDraw(gl);

         // Clear BACK BUFFER
         gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);
      }
      // draw objects with textures
      draw(gl);
   }


The solution is:

gl.glTexEnvf(GL10.GL_TEXTURE_ENV, GL10.GL_TEXTURE_ENV_MODE, GL10.GL_REPLACE);

Textures replace color settings with GL10.GL_REPLACE.
Default is GL_MODULATE.

Hi!

I'm also doing the color picking method but I get the same color for the three objects I draw. My program so far is very simple, it draws 2 squares and one triangle and draws each one with a different red-index when the user is touching the screen.

The thing is that I'm getting the same red-index for the three objects, and plus, the objects are drawn coloured (so the buffer doesn't get cleared...).

I have tried to do the things others explain above but I don't understand how to use
gl.glTexEnvf(GL10.GL_TEXTURE_ENV, GL10.GL_TEXTURE_ENV_MODE, GL10.GL_REPLACE);

Here is my code:

The onDrawFrame method (within my Renderer). Here is where I render the polygons with an unique color when the user is touching the screen.



The pickingDraw method is the following (within my Renderer too):



On the other hand, I inherited GLSurfaceView so I can handle user touches. Within this class, I use this method:



It seems that it detects the polygons when I touch them, but I get the same color index for the three of them. Plus, the three objects have the same exact color (I even drew each one with a value of red, green and blue respectively, but they ended up being drawn with the color of the last one that was drawn. For example, if the triangle is the last being drawn, and its color is blue, the three objects get drawn in blue).

Finally, this is the output I get when I touch an square, then an empty space, and another square:



I also have included the permissions mb_zi mentioned.

I don't know why the red values are the same for each object.

Any help would be appreciated.
Thanks.

Hi Pedriyoo,

Some points:
- color picking needs no permission
- Is not needed

// Translate screen coordinate Y to openGL-screen coordinates
mGLView.mPickingY = (int)(mHeight - mGLView.mPickingY);

- your pickingDraw must be the same as if you draw your object normaly, but

Code: Select all

gl.glDisable(GL10.GL_TEXTURE_2D);
gl.glDisable(GL10.GL_FOG);
gl.glDisable(GL10.GL_BLEND);
...

disabled.

This is only a solution, if you use textures:

Code: Select all

gl.glTexEnvf(GL10.GL_TEXTURE_ENV, GL10.GL_TEXTURE_ENV_MODE, GL10.GL_REPLACE);

Use it in

Code: Select all

onSurfaceCreated(GL10 gl, EGLConfig config)

Regards Harry

Color picking in 4 steps:

- clear buffers
- draw objects in unique colors (pickingDraw(GL10 gl))
- clear buffers (removes all picking objects)
- draw objects normal (draw(GL10 gl))

!!! Do in pickingDraw() the same as in draw(), but disable blending, fog, textures... !!!

Code: Select all

public void onDrawFrame(GL10 gl)
{
   // clear Buffers
   gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);
         
   if (m_Control.isPickingEnabled())
   {
      pickingDraw(gl);
               
      //clear Buffers
      gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);
   }
            
       // draw normal
   draw(gl);
}

pickingDraw(GL10 gl):

Code: Select all

private void pickingDraw(GL10 gl)
{
   ...

   // disable options
   gl.glDisable(GL10.GL_TEXTURE_2D);
   gl.glDisable(GL10.GL_FOG);
   gl.glDisable(GL10.GL_BLEND);

   // draw objects in unique colors
   for (int i = 0; i <  m_alCoverPoints.size() && i < m_alCovers.size(); i++)
   {

      Cover cover = m_alCovers.get(i);
      gl.glPushMatrix();
                  
      // set color
      gl.glColor4f(cover.getColorRed() / 255f, 0, 0, 1);
      // set object points
      gl.glVertexPointer(3, GL10.GL_FLOAT, 0, m_alCoverPoints.get(i));
      // draw elements
      gl.glDrawElements(GL10.GL_TRIANGLES, 2 * 3, GL10.GL_UNSIGNED_SHORT, m_sbIndices);
            
      gl.glPopMatrix();
   }
      
   // color picking
   gl.glReadPixels(m_Control.getPickingX(), m_Control.getPickingY(), 1, 1, GL10.GL_RGBA, GL10.GL_UNSIGNED_BYTE, m_Control.getPickingPixelBuffer());

   // tell other theads picking is ready
   m_Control.deactivatePicking();
      
   // enable options
   gl.glEnable(GL10.GL_TEXTURE_2D);
   gl.glEnable(GL10.GL_FOG);
   gl.glEnable(GL10.GL_BLEND);
}