How to render images using Cardboard App

03:00 6 Comments


This project aims at developing inexpensive virtual reality (VR) image rendering for everyone on a click of a trigger. This project enables users to know about the use of VR Cardboard and its functioning. 
Equipment Required – VR Cardboard, NFC enabled device.
How to use the above functionality for end users?
The following steps involved in the use of VR application for image zoom functionality are listed below:
Step 1: Start the application on your mobile devices.
Step 2: The Application will ask the user to put the device into the cardboard.
Step 3: Pull the magnet to see the images.
Step 4: You will find the image getting rendered on pulling the magnet each time.
How to develop this application on Eclipse?
It is very simple for developers to make this application go live. Follow the below given steps to achieve your goal:
Step 1: Click on New in the File Menu Bar.
Step 2: Go to the Android Application Project in the drop down.
Step 3: A pop-up will appear showing the Application Name in which you have to enter VRImageZoom.
Similarly, enter the project name as VRMagenet, you can also keep a different project name.
Similarly, enter the package name as com.kr.cardboard, this also you can keep with a different package name. 
Step 4: The Minimum Required SDK should be API 16: Android 4.1 (Jelly Bean), target SDK you can keep as much as you want.


Step 5: Click the Next Button thrice and Choose Blank Activity, again click on Next and enter Activity Name as VrMagenetActivity.
Step 6: Enter Layout Name as main_activity click on Finish button.


Step 7: Now, download the cardboard.jar
Step 8: Go to the manifest file to give following permissions:
Ø  <uses-sdk android:minSdkVersion="16"/> indicates that the device must be running API Level 16 (Jellybean) or higher.
Ø  <uses-sdk android:targetSdkVersion="19"/> indicates our app is targetting API Level 19 (KitKat).
Ø  <uses-feature android:glEsVersion="0x00020000" android:required="true" /> indicates that the device must support OpenGL ES 2.0 to run the demo app.
Ø  android:screenOrientation="landscape" indicates that the activity's required screen orientation is "landscape." This is the orientation you must set for VR apps. The view used by the Cardboard SDK, CardboardView, only renders on fullscreen and landscape (landscape, reverseLandscape, sensorLandscape) modes.
Ø  The setting android:configChanges="orientation|keyboardHidden" is also recommended, but not mandatory.
Ø  android.permission.VIBRATE permission is required by our demo app to make the phone vibrate to inform the user that something has happened.
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.kr.cardboard"
    android:versionCode="1"
    android:versionName="1.0" >

    <uses-sdk
        android:minSdkVersion="16"
        android:targetSdkVersion="19" />

    <uses-permission android:name="android.permission.NFC" />
    <uses-permission android:name="android.permission.VIBRATE" />

    <uses-feature
        android:glEsVersion="0x00030000"
        android:required="true" />

    <application
        android:allowBackup="true"
        android:icon="@drawable/ic_launcher"
        android:label="VRMagenet"
        android:theme="@style/AppTheme" >
        <activity
            android:name="com.kr.cardboard.VrMagenetActivity"
            android:label="VRMagenet"
            android:noHistory="true"
            android:screenOrientation="landscape" >
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>

</manifest>
Step 9: Right click on the source file à New àClass  àFile Name CardboardOverlayView à Finish.
Step 10: Now,  extends LinearLayout then create a two view left  and right.
Step 11: The following code for the CardboardOverlayView as follows:
package com.kr.cardboard;

import android.content.Context;
import android.content.Intent;
import android.graphics.Color;
import android.graphics.Typeface;
import android.util.AttributeSet;
import android.util.TypedValue;
import android.view.Gravity;
import android.view.View;
import android.view.ViewGroup;
import android.view.animation.AlphaAnimation;
import android.view.animation.Animation;
import android.widget.ImageView;
import android.widget.LinearLayout;
import android.widget.TextView;

import com.kr.cardboard.R;

/**
 * Contains two sub-views to provide a simple stereo HUD.
 */
public class CardboardOverlayView extends LinearLayout {
       private static final String TAG = CardboardOverlayView.class
                     .getSimpleName();
       private final CardboardOverlayEyeView mLeftView;
       private final CardboardOverlayEyeView mRightView;
       private AlphaAnimation mTextFadeAnimation;

       public CardboardOverlayView(Context context, AttributeSet attrs) {
              super(context, attrs);
              setOrientation(HORIZONTAL);

              LayoutParams params = new LayoutParams(LayoutParams.MATCH_PARENT,
                           LayoutParams.MATCH_PARENT, 1.0f);
              params.setMargins(0, 0, 0, 0);

              mLeftView = new CardboardOverlayEyeView(context, attrs);
              mLeftView.setLayoutParams(params);
              addView(mLeftView);

              mRightView = new CardboardOverlayEyeView(context, attrs);
              mRightView.setLayoutParams(params);
              addView(mRightView);

              // Set some reasonable defaults.
              setDepthOffset(0.016f);
              setColor(Color.rgb(150, 255, 180));
              setVisibility(View.VISIBLE);

              mTextFadeAnimation = new AlphaAnimation(1.0f, 0.0f);
              mTextFadeAnimation.setDuration(5000);
       }

       public void show3DToast(String message) {
              setText(message);
              setTextAlpha(1f);
              mTextFadeAnimation.setAnimationListener(new EndAnimationListener() {
                     @Override
                     public void onAnimationEnd(Animation animation) {
                           setTextAlpha(0f);
                     }
              });
              startAnimation(mTextFadeAnimation);
       }

       public void show3DImage(int mScore, Context context) {
              setImg(mScore, context);

       }

       public void show3DSplashImage() {
              setImgSplash();

       }

       private void setImgSplash() {
              mLeftView.imageView.setLayoutParams(new LayoutParams(
                           LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
              mLeftView.imageView.setBackgroundResource(R.drawable.magnet);
              mRightView.imageView.setLayoutParams(new LayoutParams(
                            LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
              mRightView.imageView.setBackgroundResource(R.drawable.magnet);
       }

       private abstract class EndAnimationListener implements
                     Animation.AnimationListener {
              @Override
              public void onAnimationRepeat(Animation animation) {
              }

              @Override
              public void onAnimationStart(Animation animation) {
              }
       }

       private void setDepthOffset(float offset) {
              mLeftView.setOffset(offset);
              mRightView.setOffset(-offset);
       }

       // ---------------------------------------------------------------------------------------------
       private void setImg(int mScore, Context context) {

              switch (mScore) {
              case 0:
                     mLeftView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mLeftView.imageView.setBackgroundResource(R.drawable.main);
                     mRightView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mRightView.imageView.setBackgroundResource(R.drawable.main);
                     break;
              case 1:
                     mLeftView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mLeftView.imageView.setBackgroundResource(R.drawable.image2);
                     mRightView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mRightView.imageView.setBackgroundResource(R.drawable.image2);
                     break;
              case 2:
                     mLeftView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mLeftView.imageView.setBackgroundResource(R.drawable.image3);
                     mRightView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mRightView.imageView.setBackgroundResource(R.drawable.image3);
                     break;
              case 3:
                     mLeftView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mLeftView.imageView.setBackgroundResource(R.drawable.image4);
                     mRightView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mRightView.imageView.setBackgroundResource(R.drawable.image4);
                     break;
              case 4:
                     mLeftView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mLeftView.imageView.setBackgroundResource(R.drawable.image5);
                     mRightView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mRightView.imageView.setBackgroundResource(R.drawable.image5);
                     break;
              case 5:
                     mLeftView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mLeftView.imageView.setBackgroundResource(R.drawable.image6);
                     mRightView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mRightView.imageView.setBackgroundResource(R.drawable.image6);
                     break;
              case 6:
                     mLeftView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mLeftView.imageView.setBackgroundResource(R.drawable.image7);
                     mRightView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mRightView.imageView.setBackgroundResource(R.drawable.image7);
                     break;
              case 7:
                     mLeftView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mLeftView.imageView.setBackgroundResource(R.drawable.image8);
                     mRightView.imageView.setLayoutParams(new LayoutParams(
                                  LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
                     mRightView.imageView.setBackgroundResource(R.drawable.image8);
                     break;
              default:

                     Intent intent = new Intent(context, VrMagenetActivity.class);
                     context.startActivity(intent);

              }

       }

       // ------------------------------------------------------------------------------------------

       private void setText(String text) {
              mLeftView.setText(text);
              mRightView.setText(text);
       }

       private void setTextAlpha(float alpha) {
              mLeftView.setTextViewAlpha(alpha);
              mRightView.setTextViewAlpha(alpha);
       }

       private void setColor(int color) {
              mLeftView.setColor(color);
              mRightView.setColor(color);
       }

       /**
        * A simple view group containing some horizontally centered text underneath
        * a horizontally centered image.
        *
        * This is a helper class for CardboardOverlayView.
        */
       private class CardboardOverlayEyeView extends ViewGroup {
              private final ImageView imageView;
              private final TextView textView;
              private float offset;

              public CardboardOverlayEyeView(Context context, AttributeSet attrs) {
                     super(context, attrs);
                     imageView = new ImageView(context, attrs);
                     imageView.setScaleType(ImageView.ScaleType.FIT_CENTER);
                     imageView.setAdjustViewBounds(true); // Preserve aspect ratio.
                     addView(imageView);

                     textView = new TextView(context, attrs);
                     textView.setTextSize(TypedValue.COMPLEX_UNIT_DIP, 14.0f);
                     textView.setTypeface(textView.getTypeface(), Typeface.BOLD);
                     textView.setGravity(Gravity.CENTER);
                     textView.setShadowLayer(3.0f, 0.0f, 0.0f, Color.DKGRAY);
                     addView(textView);
              }

              public void setColor(int color) {
                     // imageView.setColorFilter(color);
                     textView.setTextColor(color);
              }

              public void setText(String text) {
                     textView.setText(text);
              }

              public void setTextViewAlpha(float alpha) {
                     textView.setAlpha(alpha);
              }

              public void setOffset(float offset) {
                     this.offset = offset;
              }

              @Override
              protected void onLayout(boolean changed, int left, int top, int right,
                           int bottom) {
                     // Width and height of this ViewGroup.
                     final int width = right - left;
                     final int height = bottom - top;

                     // The size of the image, given as a fraction of the dimension as a
                     // ViewGroup. We multiply
                     // both width and heading with this number to compute the image's
                     // bounding box. Inside the
                     // box, the image is the horizontally and vertically centered.
                     final float imageSize = 1.0f;

                     // The fraction of this ViewGroup's height by which we shift the
                     // image off the ViewGroup's
                     // center. Positive values shift downwards, negative values shift
                     // upwards.
                     // final float verticalImageOffset = -0.07f;
                     final float verticalImageOffset = -0.00f;

                     // Vertical position of the text, specified in fractions of this
                     // ViewGroup's height.
                     final float verticalTextPos = 0.52f;

                     // Layout ImageView
                     float imageMargin = (1.0f - imageSize) / 2.0f;
                     float leftMargin = (int) (width * (imageMargin + offset));
                     float topMargin = (int) (height * (imageMargin + verticalImageOffset));
                     imageView.layout((int) leftMargin, (int) topMargin,
                                  (int) (leftMargin + width * imageSize),
                                  (int) (topMargin + height * imageSize));

                     // Layout TextView
                     leftMargin = offset * width;
                     topMargin = height * verticalTextPos;
                     textView.layout((int) leftMargin, (int) topMargin,
                                  (int) (leftMargin + width), (int) (topMargin + height
                                                * (1.0f - verticalTextPos)));
              }
       }
}
In the above code, we have extended the linear layout and created a function with function name show3DImage to make the view more clear and visible.
Step 12: The Cardboard provide its own view named as CardboardView, which is a convenience extension of  GLSurfaceView that is used for VR rendering. Now, you can see how to define a CardboardView in the activity layout xml file along with our CardboardOverlayView in the following way:
<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/ui_layout"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    android:orientation="vertical" >

    <com.google.vrtoolkit.cardboard.CardboardView
        android:id="@+id/cardboard_view1"
        android:layout_width="fill_parent"
        android:layout_height="fill_parent"
        android:layout_alignParentLeft="true"
        android:layout_alignParentTop="true" />

    <com.kr.cardboard.CardboardOverlayView
        android:id="@+id/overlay"
        android:layout_width="fill_parent"
        android:layout_height="fill_parent"
        android:layout_alignParentLeft="true"
        android:layout_alignParentTop="true"
        android:layout_centerInParent="true" />

</RelativeLayout>
Step 13: Open the class VrMagenetActivity as defined below:
CardboardActivity is the starting point for coding a cardboard app. CardboardActivity is the base activity that provides easy integration with Cardboard devices. It exposes events to interact with Cardboards and handles many of the details commonly required when creating an activity for VR rendering.
Note that CardboardActivity uses sticky immersive mode, in which the system UI is hidden, and the content takes up the whole screen. This is a requirement for a VR app, since CardboardView will only render when the activity is in fullscreen mode. See Using Immersive Full-Screen Mode for more discussion of this feature.
The demo app's VrMagenetActivity extends CardboardActivity. MainActivity implements the following interface:
CardboardView.StereoRenderer: Interface for renderers that delegate all stereoscopic rendering details to the view. Implementors should simply render a view as they would normally do using the provided transformation parameters. All stereoscopic rendering and distortion correction details are abstracted from the renderer and managed internally by the view.
Step 14: Initializes the CardboardView in the onCreate() method:
/**
* Sets the view to our CardboardView and initializes the transformation matrices we will use
* to render our scene.
* @param savedInstanceState
*/
@Override
public void onCreate(Bundle savedInstanceState) {
   super.onCreate(savedInstanceState);
   setContentView(R.layout.common_ui);
   CardboardView cardboardView = (CardboardView) findViewById(R.id.cardboard_view);
   // Associate a CardboardView.StereoRenderer with cardboardView.
   cardboardView.setRenderer(this);
   // Associate the cardboardView with this activity.
   setCardboardView(cardboardView);

   // Initialize other objects here.
}
Once you get the CardboardView you associate it with a renderer, and then you associate         
the CardboardView with the activity. 
Step 15: Now define CardboardOverlayView class as-
CardboardOverlayView mOverlayView =(CardboardOverlayView)findViewById(R.id.overlay);
Step 16: Now, how to use magnet –To provide custom behavior when the user pulls the magnet, override CardboardActivity.onCardboardTrigger() in your app's activity. Call the show3d Image.
@Override
       public void onCardboardTrigger() {

              Log.i(TAG, "onRendererShutdown1");
              mOverlayView.show3DImage(mScore++, VrMagenetActivity.this);
              // mScore++;

              mVibrator.vibrate(50);
       }

Now, please go through the whole code as mentioned below:
package com.kr.cardboard;

import javax.microedition.khronos.egl.EGLConfig;
import android.content.Context;
import android.net.Uri;
import android.os.Bundle;
import android.os.Vibrator;
import android.util.Log;
import android.widget.ImageView;
import com.google.vrtoolkit.cardboard.CardboardActivity;
import com.google.vrtoolkit.cardboard.CardboardView;
import com.google.vrtoolkit.cardboard.EyeTransform;
import com.google.vrtoolkit.cardboard.HeadTransform;
import com.google.vrtoolkit.cardboard.Viewport;
import com.kr.cardboard.R;

/**
 * A Cardboard sample application.
 */
public class VrMagenetActivity extends CardboardActivity implements
              CardboardView.StereoRenderer {
                     
       private static final String TAG = "MainActivity2";

       Uri fileUri = null;
       ImageView photoImage = null;

       private int mScore = 0;

       private Vibrator mVibrator;

       private CardboardOverlayView mOverlayView;

       /**
        * Sets the view to our CardboardView and initializes the transformation
        * matrices we will use to render our scene. //@param savedInstanceState
        */
       @Override
       public void onCreate(Bundle savedInstanceState) {
              super.onCreate(savedInstanceState);

              setContentView(R.layout.main_activity);
              CardboardView cardboardView = (CardboardView) findViewById(R.id.cardboard_view1);
              cardboardView.setRenderer(this);
              setCardboardView(cardboardView);

              mVibrator = (Vibrator) getSystemService(Context.VIBRATOR_SERVICE);

              mOverlayView = (CardboardOverlayView) findViewById(R.id.overlay);

              mOverlayView.show3DSplashImage();

       }

       @Override
       public void onCardboardTrigger() {

              Log.i(TAG, "onRendererShutdown1");
              mOverlayView.show3DImage(mScore++, VrMagenetActivity.this);
              // mScore++;

              mVibrator.vibrate(50);
       }

       @Override
       public void onRendererShutdown() {
              Log.i(TAG, "onRendererShutdown");
       }

       @Override
       public void onSurfaceChanged(int width, int height) {
              Log.i(TAG, "onSurfaceChanged");
       }

       /**
        * Creates the buffers we use to store information about the 3D world.
        * OpenGL doesn't use Java arrays, but rather needs data in a format it can
        * understand. Hence we use ByteBuffers.
        *
        * @param config
        *            The EGL configuration used when creating the surface.
        */
       @Override
       public void onSurfaceCreated(EGLConfig config) {
              Log.i(TAG, "onSurfaceCreated");
       }

       /**
        * Prepares OpenGL ES before we draw a frame.
        *
        * @param headTransform
        *            The head transformation in the new frame.
        */
       @Override
       public void onNewFrame(HeadTransform headTransform) {
       }

       /**
        * Draws a frame for an eye. The transformation for that eye (from the
        * camera) is passed in as a parameter.
        *
        * @param transform
        *            The transformations to apply to render this eye.
        */
       @Override
       public void onDrawEye(EyeTransform transform) {
       }

       @Override
       public void onFinishFrame(Viewport viewport) {
       }

}

Delhi, INDIA

6 comments:

  1. Unknown23 November 2015 at 15:31

    Hello,

    Thank you very much for this useful blog.
    I wonder if I could use the sample code above and leverage it with street view (thing that google has already done).
    Actually I am quite novice in Cardboard ans Google Maps programming but would it be right to use that sample code and inject Street View binocular images in each eye to get the 3D rendering?
    Do you know maybe if Google provides an API that is doing the job? Or any sample code?

    Thank you very much for your help and for that precious blog.

    Regards,

    Fabrice Saadoun.

    ReplyDelete
    Replies
    1. Unknown30 March 2016 at 00:14

      Hi. I would also wish to apply the same effect to my app. Please let me know if you were able to find a solution to your problem :)

      Delete
  2. Unknown10 March 2016 at 20:49

    Can you please provide us link to download this project?
    thanks!

    ReplyDelete
  3. Unknown30 March 2016 at 00:09

    Thanks for this. Really Helpful. Could you please upload to GitHub because i still couldn't get it to work

    ReplyDelete
  4. Yasir Tahir19 June 2016 at 15:02

    What is magnetic pull? Will it work with a cardboard without magnetic input? If not, how can we make it work without it.

    ReplyDelete
  5. David Russel4 July 2017 at 12:17

    Wonderful piece of information. Well, You can either choose some of the best VR box apps for android Although, if somebody not familiar with these codes can use prebuild app on their devices.
    Thanks

    ReplyDelete

Subscribe to: Post Comments (Atom)