Archive
CrimeRunner the AR game I was working on 2011 does not sound like a bad idea now does it
I just stumbled on this game I was working on back in 2011. With no idea on how to create a mobile game. The artwork is not mine. I got images from the internet and used them to make a story. If anyone know who the artist is let me know to give him credit.
Back in the days everyone told me this was not worth it and that it would not change health. Well obviously it did not because I could not execute it but I believe the game mechanics and dynamics just like my other post (game) Eferteon makes complete sense. Just wanted to share
Don’t like walking? PokemonGO could change that for you
For several years now different companies have been creating location based games. These games use our smartphones GPS and integrate surrounding landmarks to later transform them into a digitally augmented reality (AR) worlds.
As Physicians we always recommend our patients to become as healthy as possible and we emphasize the words “lifestyle modifications”. Lifestyle modifications aim to change an individual’s way of living and become healthy as possible without or with the minimum possible use of medications. With lifestyle modifications we aim to reduce weight, become healthier and therefore directly impacting the course of diabetes, hypertension and cholesterol.
As a digital health innovator and video-gaming enthusiast I have always believed that gamification is the key to lifestyle modification. Gamification is defined as the application of typical elements of game playing to other areas of activity. PokemonGO is probably the best executed game and successful augmented reality game that is indirectly impacting the health of individuals by making them get out, walk and explore their neighborhood.
PokemonGO was created by Niantic Labs who are also the creators of another augmented reality game called Ingress. The reason why this is relevant is that all those Pokestops and Pokegyms that you see are an importation of the database from ingress. The Ingress portals that were created by people playing the game and taking pictures of locations are now the pokestops and pokegyms.
The reason why I am excited about PokemonGo and future AR games is that this game has arguably managed to tackle the “Lifestyle Modification realm” with far greater impact than the whole medical community placed together in the last 50 years.
We always tell our patients to exercise at least 30 minutes a day, to eat healthy, to lose weight, avoid becoming obese. In most cases with little impact. The reason why this happens, lets face it, is that only a lucky few really get pleasure from exercise. If exercise is not fun it turns into a chore, and if it is a chore we will not keep doing it. Nobody wants to come home from work just to get into another chore. For those of you that can do it congratulations, for those of you who find it more challenging AR games might have finally found a right formula.
I have tried multiple apps/games in the past that have the potential to modify our lifestyle. To name a few; Life is Crime, Life is Magic, Zombies Run, Ingress and more recently I have been keeping my eye on one called Run an Empire, this last one looks great but has not been released.
So what is the big deal about PokemonGO?
Simple the average video gamer age is now 38 years old. Yes you heard it, this means we grew up with Atari, Colecovision, Sega, Nintendo, Snes, Nintendo 64, Xbox, Playstation, Steam. Personally I am currently exploring Virtual Reality as well with the Oculus Rift, HTCVive, Sony Morpheus. In any case, the important thing here is that Niantic Labs partnered with Nintendo (who own majority stake of Pokemon Co) and they have managed to create a game that allures kids of all ages (including those in there 70s). I have been using the game since its and almost Level 21 and I can tell you I am extremely excited about it. Never in my life have I seen a crowd of people walking around in groups. In the past, people chasing and trying to capture imaginary characters would have been considered crazy, now they are our society.
I have been going to different locations witnessing an ever-growing group of people walking around. At first I must admit it feels awkward, to a certain extent because you feel you are being judged and thus might hold back on using it. My recommendation is don’t. Never stop doing things because of what other might think of you. If you want to ahead and try it GO AHEAD. The first time I used it in public was in South Padre Island, I went for a run and started capturing Pokemon and started noticing other people staring and walking with their phones. I started asking people if they were playing PokemonGo and they started laughing “YES! Are you?!” Me “Definitely!”. This continued and I have met many people in similar scenarios. I would like to emphasize a PAUSE here! Yes PokemonGO has also been targets of assaults and other crimes, it is not a reason on why not to use it, just be careful and use proper judgement please.
I continued to play PokemonGo and started exploring Mission. In order to find more Pokestops and Pokegyms it occurred to me to use the Ingress Intel site (previous game of Niantic whose database was imported) to find more locations. One key region I have the Veteran’s War Memorial. The VWM has in a 5 Pokestops and 5 Pokegyms together which has managed to create crowds that are growing in a daily basis. Not to mention that it is also likely increasing sales.
Initially it was the younger population, mostly teens but this has changed dramatically in the last couple of weeks and many videos have been uploaded to youtube in which hundreds if not thousands of people are walking/running chasing PokeMons. What I have been witnessing lately is that parents of children are now taking their children for walks. Hey, be it what it may be, people are walking now and that is a Start!
As an individual replied to the post I was interviewed for in TCTMD. As funny as that sounded to me at first, he’s right. My initial reaction was, “Well yeah, of course Pokémon fans are going to be into it.” But then I remembered that my 74 year old dad made me install it on his phone a week ago and he’s still super psyched about it. He grumbles when the servers go down and before Pokémon GO he had no idea what Pokémon really was. And now we take long walks together every week, fully stocked with a 20,000 mAh power bank and 2 liters of water.
Due to the popularity of Pokemon, it is hard to believe that this hype will be weaning, on the opposite it will continue to grow. I can easily see how this can be incorporated to promote healthier lives in McAllen from Schools to Nursing homes. For those that think this is an unrealistic comment I would just like you to know that C.S. Mott Children’s Hospital in Ann Arbor, Michigan is using the popular game to get kids out of their rooms so they can exercise and be social with other young patients. It is just about thinking outside the box, something the Medicine has been needing for many years now.
For those of you playing the game or wanting to get started on it I will leave some pointers on how to level up your character quickly to catch ’em all. 1) Error #1 people don’t walk as much as they should around pokestops. I have managed to level up my character by walking in a fast pace and running in safe places while playing the game. It is a skill to be mastered and yes accidents can happen but so can they when you are mountain biking but there are no Mountains in McAllen! I now can run and catch Pokemon while running with certain ease. If you are concerned about running out of Pokeballs well then run near Pokestops so you can replenish your equipment 2)If you find a place where there are a lot of Pokestops like the VWM, use a lucky egg, + incense + take advantage of all the lures that people place and this will surely help you level up quickly, I am now level 21 3) DO NOT waist stardust leveling up a character at least until you are level 15. Save it and when you are a higher level you will be able to capture higher tier Pokemon without having to waist the stardust.
In a near future RGV Cardiology will be holding events to help you level up your character and get healthy at the same time! See you then and happy hunting!
PokemonGo is the app to help us Tackle Obesity
Gamification, in order to get us healthier, is not a new concept but up until now the games have failed to lure people to exercise. Pokemon GO has it right in my opinion. The reason why this is true is that your average gamer is now 38 years old and we grew up with this game. In addition, Pokemon has also lured the younger population as well.
In addition this is exactly what I meant when I made the application for Google Glass. I tried to incorporate into it a game and I designed Eferteon into it but proved to be too challenging for its execution. In any case, right direction for health. Below is what I had in mind in 2013. PokemonGO WILL EVOLVE to be used with technology like GoogleGlass mark my words.
So PokemonGo is out and people are walking searching for Pokemon everyewhere. THIS IS NOT FOR KIDS ONLY. Adults should be playing with it and even business are realizing that they can use it to monetize in certain ways. (Not getting into details on how).
For those that do not know. PokemonGo comes from collaboration between Nintendo and Niantic Labs. Niantic created incress and Pokemon is basically using data acquired from ingress to create pokestops and other landsmarks. It also shows us the way that digital data is being used. (wait till pokemon incorporates sensors). I chose team Mystic which is blue.
I have been walking now and I am excited to see people everywhere stoked about it. Asking about their team, levels, how to lure rare pokemon. Below are a couple of guys I met at SPI.
In addition I have also managed to get my daughter to walk with me in this hot weather 😉
Get on board now. It does not matter your age. Personally as a Cardiologist, I see myself prescribing this to my patients. NO I am not kidding! Happy Hunting
Impact of Wearable Technology will be linked to GOOD Gamification
We are finally witnessing wearables grabbing the attention of people (MDs, Patients, you name it) in order to improve health. I believe wearables have tremendous potential and will provide very important data but the the true question will be -> How to make people want to continue using the wearable? Sales for wearables is strong, but users are abandoning them quickly after
Just to keep data will be as enticing as using the weight scale in a daily basis. I have always said Gamification is the most important “glue” for their compliance.
I believe that Augmented Reality games have the potential to get people outside once again and explore the world. Pokemon go is such an application and everything points it will come out next week. Pokemon Go comes from the collaboration of Niantic Labs with Nintendo. They are the creators of Ingress AR Game, and they have applied similar concepts to the Pokemon GO game.
Now their Pokemon GO plus is not a wearable like the fitbit but nevertheless it could motivate people to walk. In addition it will have in app purchases it seems and here is the kicker from the cardiovascular standpoint.
I hope that one day In app purchases could be done by transforming caloric burn into in app currency instead of real money. Thus the more calories you burn the stronger your character will become.
In the past I tried to create Eferteon (2012, still a cardiology fellow), a game that focused on this concept. Secondary to demanding Medical schedule I was never able to concretize it. Maybe someone can get ideas from this and apply it. IT WILL SHOW HOW WEARABLES COULD BE USED TO MOTIVATE PEOPLE TO EXERCISE
DO NOT FOCUS ON WEARABLES FOR SICKCARE!!! FOCUS ON WEARABLES FOR HEALTH-CARE! If you focus on sick care we will wait years to see research that will contradict with each other. MEH!
IN THE CASE OF EFERTEON. I OBTAINED THE IMAGES FROM INTERNET TO ILLUSTRATE MY CONCEPT. The game was never created but it illustrates how I think that wearables have the potential to improve health by quantifying caloric effort and transforming to usable in app/game currency.
TeleRobotic Procedures via VR Cockpit are closer than you think
Back in 2011 I had the opportunity of assisting Singularity University’s first FutureMed (now known as Exponential Medicine). I was exposed to several exponential technologies. Relevant to this post was the talk given about Driverless Cars by Brad Templeton. Back in 2011, in general, people said this was not feasible and it would take more than 20 years for this to happen. Well , I believe everyone now understands how deceiving exponential trends can affect our daily life.
In the last couple of years I have focused in the implementation of Virtual Reality in the cathlab and was the first one to record with Jaunt VR some procedures. Once I reviewed the recorded 360 experience I understood the wide array of possible applications and implications. I have mentioned medical education, patient education, decreasing stress and anxiety, decreasing arrhythmias in the hospital setting and one that seems more far away the Virtual Reality Cockpit which could guide telerobotic procedures.
This last month 2 very important research studies have been released:
1) The CORA-PCI Study (presented at SCAI) evaluated the feasibility of performing PCI in patients with complex coronary lesions using robotic technology to remotely control coronary guidewires and stents. Dr. Mahmud showed how at this point in time robotic technology to perform PCI is a viable alternative to manual PCI. Key words are AT THIS POINT IN TIME. As a technology enthuthiast I guarantee you this will change to be the norm and will allow us to perform intervention in a quicker and safer manner. (I know many will think this is nonesense, but everything I have been saying since a medstudent regarding implementation of technology has eventually happened and always considered nonesense. So let me just say this will happen)
2) As A VR pioneer and enthuthiast I find the following study tremendously exciting. Even more exciting is that this comes from @DisneyResearch which I just find it as a breath of fresh AIR! The study is “A Hybrid Hydrostatic Transmission and Human-Safe Haptic Telepresence Robot”
The latter is not directly related to medicine but this just proves that what I said at @CRT_Meeting is true. (Which I managed to record in 360 immersive video, pardon the youtube quality) The Virtual Reality Cockpit is coming and it will allow us to perform TeleRobotic Procedures.
All you need to know to capture 360 “Virtual Reality”
I have been a strong advocate of Virtual Reality as well as Augmented Reality since 2011 and technology has allowed me to truly experiment with it since late 2014.
As an interventional Cardiologist I looked for ways to implement it in the Cathlab for Medical Education. I did this initially thanks to to the help of JauntVR and Medtronic. You can check the post in Nuviun here – “Interventional Cardiologist uses Virtual Reality in the Cathlab”
Since I wanted to be able to do this on my own I searched for many different options to do so, it seemed that before and for the time being the best way to do it is with 3d Printed Virtual Reality Rigs and modified GoPro cameras. I was able to do this thanks to the help of Virtual Natives (Javier Herreros and Cristian Dominguez) they are VR Masters
So this is step #1 – Creating your RIG. First thing I would recommend is to modify you GoPro Camera and add a larger FOV lense. Such as the picture elucidates below. I started making one from Epoxy Putty then continued with 3D Printing Technology. In addition you will need Entaniya Lenses or Izugar Lenses
If you have purchased the lenses here is a video on how to remove them. This is for the Izugar ones which I use.
And once you have done it and if you have a 3d Rig it should look like the picture above the video. But DO not sweat it. If you do not have a 3D rig I also found that your hands are the cheapest 3D printers Available. And yes an Epoxy Putty Rig works well 😉 Picture…
Ok so you have your rig, now what? Well you will need a MonoPod that is slim and that will not interfere with the images of you Cameras. I find this one to be a great one Polaroid 65″ VariPod 2-in-1 Telescoping Camera Monopod
An important point of VR, no matter of how many different cameras you have is that you have to synchronize them to start recording at a same time, not imperative, but it does help in the stitching of the videos and for this you will need a GoPro Remote controller so you can start recording in all the cameras at the same time.
Step #2: You will need to “Stitch” the Videos and then likely edit them with another software. In order to stitch the videos you will need Ideally Autopano Video but Autopano Giga can also help in challenging stitching scenarios.
Here is an Example of what Autopano Video looks like:
I learned to use this software by trial and error. It was a little painful but I had fun doing so. There are some tutorials in the net but this in my opinion was the most challenging step. These days it is easier because there are many presets included that make stitching very easy. If you need help with this step contact me or contact Virtual Natives they will be happy to help.
Once you have your stitched Video then comes the editing portion. Yes this is elective but makes your video look better and its fun 😉 I used Adobe Premiere for this and I learned how to use it with online course at UDEMY
Once you have your videos you will be able to share with the world your experience. Now starting with education will be the most simple step but I will elucidate how this will play in the hospital briefly
- It will make Hospital stays better, possible shorter, and improve patient satisfaction and outcome. (How you may ask, I will show you later)
- It will enable the patients to interact with family members like it happens in the app VTIME . (Yes I have used it. I will not talk about something if I do not understand it and I have not experienced it)
- It will improve in Cardiac Rehabilitation and Post-Operatory State
- It will reduce patient anxiety
- Many more things but this is not the point of this post
- Convergence of VR with Robotics and you will have a VR Cockpit. I have said this in the past in some of my conferences already…
From my experience sharing them in Youtube is ok but the quality, well how can I say this scientifically… SUCKS and KOLOR Eyes is by far better. In addition you can zoom in and out.
Here is a very quick example of it being used in the Cathlab. This I just did quickly in literally 5 minutes, improvising the camera position and it turned out to be decent. The quality is i good since I uploaded it to KOLOR.
Below, is the first 360 Immersive VR conference. This will be the way we attend medical meetings in a future. I recorded it myself, processed and edited with the techniques mentioned above. The quality is reduced secondary to youtube platform but at least you get the idea. By the way it is a great talk 😉 It happened in Cardiovascular Research Technologies
With this one you Can use Google Cardboard if you have it!
Last but not least, this following slides will summarize some important key points.
This process will be simplified immensely in 2016. Personally I am truly excited about this Samsung Camera or the Vuze
There you have it! Questions, feel free to send them via twitter @ChristianAssad
Create your own Virtual Reality Rig with Epoxy Putty and get rolling!
I have been experimenting for a while with 360 Immersive Virtual Reality. Initially I was able to try it out in the Cathlab thanks to the help from Jaunt VR. Here is the post of what we did: “Using Virtual Reality in Interventional Cardiology”
For me Virtual Reality is an exciting field that needs to be incorporated into medicine in a quicker pace. This will happen from doctors, residents, medstudents, PAS, nurses, you name it, familiarizing with it and experiencing it.
Well in the past it was challenging to come up with your own system. Multiple cameras, a 3D printer and good luck. So I started exploring what would be the easiest way in order for everyone to be able to do it and I think I have found it
Materials Needed!
- 2 Modified GoPros with 2 Fisheye lenses
- Your hands and Epoxy Putty… Yes YOU BECOME THE 3D PRINTER 😉
Yes there is a little more to it but if you are enthusiastic like I am you will be able to create some pretty AWESOME stuff. Stay tuned and you will see!
Here are some Pics
I) After mixing the SteelStick and molding it.
II) I created a top and bottom base but soon realized I did not have to. A bottom base is enough. Ye ye ye it does not look pretty but it WORKS!
III) This is as minimalistic as it gets and works like a charm.
Click on the Image below and you will get a feel of what you can do!
C
Time to experiment with #VirtualReality! What you need to know!
* Note: I give a small introduction for those unaware of AR/VR. If you are here to find out about options to start recording experiences for VR jump to the last paragraph 😉
As Moore’s law continues to predict the tremendous advancements in technology, our imagination and creativity is becoming the essential catalyst in converging them to target particular goals. In order for people to understand the potential of Moore’s law I recommend for people to imagine where they would be in 30 linear steps or in 30 exponential steps. Here is a slide I made.
One of the beauties of Innovation is that exciting ideas arise when we mix different fields. A perfect example of such is Pumps & Pipes. P&P is an association of Medical, Energy, Aerospace, Academic, Community professionals and leaders that brings together professional groups who may not otherwise have the opportunity to interact for the transfer of knowledge and technology knowhow.The ability to see beyond a particular profession and understand that we can incorporate methods, processes, theories of different ones into our own is what ignites the innovation spark.
When Google Glass came out, it was called an explorer program for a reason. It was meant to allow particular dreamers to use the technology in different ways. In my case I quickly applied it to the Medical Realm. Using augmented reality to help bystanders give efficient CPR we created “CPRGLASS” A few months later, we demonstrated that potential it had in TeleMentoring medical procedures. “Visionary–Google Glass Moves From Concept To Clinical Care”.
Since then I have been trying to incorporate Augmented Reality in different settings. Experimenting with different smartglasses (Atheer Labs, META glasses, ODG) has been important and each one has its strengths and weaknesses. Without a doubt there is tremendous potential in Augmented Reality but technology is moving in a very fast pace and therefore choosing a technology that has the least resistance for implementation is crucial to make the innovation process more exciting and less of a hurdle jumping scenario.
This is the reason why focusing on Virtual Reality was an intuitive approach. The concept of AR/VR is not new and has been explored since 1968 with the creation of “The Sword of Damocles” by Ivan Sutherland. Since then, it has been following Moore’s law which has allowed a device that was so heavy that needed to be suspended from the ceiling to a normal pair of glasses. Below is “The Sword of Damocles” (and you thought glass looked weird)
There are many different options of VR headset and they will continue to grow. (This is a good List) In addition eye tracking options are now being created and FOVE is one of them (still in Kickstarter and met funding goals in less than 3 days)
How can we use Virtual Reality to improve Patient Care?
Many options exist regarding applications of Virtual Reality in Healthcare; Psychology/Psychiatry, Meditation, Medical Education. As someone who enjoys videogames (by the way your average gamer is 35 years old now) I have always dreamed of incorporating a fun experience to learning and Virtual Reality, in my opinion is just the way to do it. Initially it would be easier to record it and in the future to create a Computer Generated Interphase that we could interact with. This is why I explored recording a 360 immersive experience with the help of Jaunt VR to film 2 cases. More on this can be found here “Using Virtual Reality in Interventional Cardiology”
As I presented the findings people have asked how it is that they could create such videos. Well, Moore’s law is allowing everyone to do this now and a lot of news has been unraveled with the last Google I/O conference. Relevant to the subject is the Google Jump. Jump is an entire ecosystem for creating virtual reality video. As Clay Bavor, Google’s vice president of product, detailed on stage Jump consists of three parts: 1) the camera rig itself 2) software that automatically assembles and processes the footage, and 3) the player.
Virtual Reality now has the attention of a lot of people since this has happened and many have been asking me ways to capture 360 immersive experiences since not many have the opportunity have work with a company like Jaunt VR (which I am very grateful).
With the recent acquisition of Kolor by GoPro it seems that the consumer approach to capture Virtual Reality relies on using several GoPro cameras and it is not as expensive as you may think.
Options
- Google Jump Rig
- Freedom 360
- 360 Heros
- Below is what a Setup could look like
Once the setup is assembled, platforms like jump will allow consumers to stich and assemble the footage to later share with the world. The beauty of this is that experiencing the footage will be accessible to millions of people. Basically anyone with a smartphone will be able to do so if they have something like GoogleCardboard.
The impact that this can have on different fields is only limited by the individual’s creativity. At the moment I am exploring this for medical education, but as we start exploring and the technology continues to evolve we will see the convergence of this technology with ubiquitous networking, robotics, sensors and haptic feedback which will initially enrich our learning experience and in the future could potentially have a keyrole in medical interventions/procedures.
Talking about Convergence, I am beginning to explore the capacity of EchoPixel. It is an amazing empowering technology that goes hand in hand with Virtual Reality and Augmented Reality. More about it can be found in this Wired article “Holograms Will Let Doctors See 3-D Views of Our Insides” as well as this one published in Newsweek “Bringing Virtual Anatomy to the Operating Room”
My recommendation to everyone would be get GoogleCardboard now. It is available for both Android and iOS. Once you experience it then you will understand the potential and depending on your profession you might find exciting uses for it which are now becoming widely available! Good Luck!
Relevant to this is an interview with Michael Gibson on AR/VR
Important keypoints learned from using #VirtualReality in the Cathlab
Since the release of the Google Glass explorer program in 2013 different approaches for their implementation in medicine have been explored by medical pioneers. Over the last 2 years we have witnessed a boom in different augmented reality technologies. In addition we also have witnessed the rise of virtual reality.
While augmented reality (AR), in a nutshell, is the overlay of digital content into one’s field of view virtual reality (VR) completely replaces the individual’s field of view. VR is a powerful brainhack that can lead the individual’s brain to believe he is in a completely different location. It is also interesting to mention that we are activating our brains in ways we have never done before and scientists are currently exploring this realm.
A common denominator that I have found regarding the implementation of new technologies into whichever field, is that an individual needs to experience it to understand what it can truly do, what it can become and then how to use it in his profession or environment. When I initially tried to describe the augmented reality experience to the medical community, words never did justice to it. Once they experienced it, the most common reaction was “Now I get it”. The same concept applies to Virtual Reality, but in my opinion in a far more drastic manner. One thing is to add content into one’s field of view and the other is to completely shock the brain’s conception of environment and teleport it into another. This Virtual environment has now become the “true” environment for a period of time.
Shortly after I had the opportunity to explore Virtual Reality I realized the enormous potential it had in clinical practice. From potential therapeutic implications in the psychiatric realm, heart failure and cardiac rehabilitation to medical education. For this reason I decided to find an accessible way to start exploring and working with such. The most convenient option at the time was using the Gear VR Innovator edition for Note 4.
Depending on one’s environment, location and network, a different approach to how they should use it will vary. In my case, as an interventional cardiologist, I have seen training courses in which physicians travel form different states, even countries to a particular location to learn new procedure and techniques. For this reason I saw a window of opportunity in such space. If we learn and become experts by repetition, Virtual Reality is just the perfect setting for this. The tangents that can emanate from such are vast, to the point we could potentially add haptic feedback for additional tactile sensation incorporation into the learning experience. In a future, even telementoring-telerobotic applications.
In order to test the hypothesis that Virtual Reality will have an important role in the future of medical education I researched ways and techniques to capture a 360 immersive video during particular procedures. After exploring ways and techniques to do so I was lucky to be introduced to Jaunt VR. Needless to say they automatically understood the potential of how this could help medical education and ultimately improve patient care. We explored two Scenarios 1) The ultrasound guided initiative, priceless in deciding approach and exit strategy in particular interventions 2) Preparing and deploying the Medtronic CoreValve.
Once the filming was done, I had an enthusiastic expectation of what the outcome would be but I was speechless after witnessing it in VR. I had the opportunity to give a keynote at the Society for Coronary Angiography and Interventions (SCAI) in San Diego in May 6 regarding potential in implementing exponential technologies into medicine, and this recent Virtual Reality was one of them. Trying to explain and transmit these findings as well as other emerging technologies with words is close to impossible. It is like trying explaining sound to someone with hearing impairment. The brain perceives Virtual Reality in a different way than our true reality, and when an individual perceives it “awakens” a new realm and with it the potential. In order for me to relay my message in the most efficient manner, I decided to take the Gear VR to the meeting in order to get interested individuals to test the experience and understand what the words meant. From a total of 20 individuals (most MDs) who have experienced the cases with the VR Headset, their response was always a “WOW” like reaction followed by “This is amazing” and the “Now I get it”.
By understanding the implication of being able to re-live a procedure as a secondary operator, attendees quickly understood the potential in medical education. For those that did interventional procedures, they seamlessly understood how this could help in acquiring and learning new techniques.
To our knowledge, this was the first time that true 360 immersive technology has been used and incorporated to Oculus Rift type technology. The proof of concept and acceptance by the Interventional Community at SCAI was a unanimous positive one which will lead to future projects later to be announced.
Interventional Cardiologist Explores Virtual Reality in Medical Procedures with Jaunt VR
There are several reasons why I believe virtual reality will be an extremely disruptive technology in medicine, and it will start with education. In order for people to understand why this is the case they first need to understand what exactly virtual reality is. I would describe virtual reality as a technology that allows the individual to ‘teleport’ to a different environment as an invisible spectator. Effective VR is a ‘brain hack’ so strong that it can actually trick the user into believing they are someplace else, which potential leads to interesting therapeutic applications1. A virtual reality scene can be either computer-generated or captured with 360º cameras. Both approaches have tremendous potential for medical applications.
When I experienced Virtual Reality for the first time, the first thought that came to mind was the tremendous potential of such technology in medical education. For this reason I decided to explore possibilities to capture an immersive 360 environment. Shortly after my search began I quickly realized the task was more daunting than I initially expected. After evaluating the options one quickly stood out from the rest. Jaunt VR, a startup in Palo Alto, CA developing 360º video technology and considered by many as the leader in cinematic VR production. According to their senior engineer, Adam Somers, “Jaunt VR develops end-to-end solutions to enable the capture, production, and distribution of cinematic VR content. There is tremendous potential to apply live-action VR recordings to a number of industries, including medicine. Our collaboration with Dr. Assad represents an important first-step in the exploration of immersive media for educational and therapeutic applications.”
On March 25, 2015 at El Camino Hospital, Mountain View, California we explored the potential of virtual reality by capturing important teaching pearls in interventional procedures that could potentially lead to better patient care and outcomes.
- The Ultrasound Guided Access initiative. This I learned in my advanced peripheral endovascular fellowship with Dr. James Joye and has proven to be a very valuable technique that without a doubt can avoid disastrous complications if used properly.
- With the assistance of the Medtronic team we were able to setup up a simulator to evaluate feasibility. We explored capturing the steps involved in the preparation and deployment of a Medtronic CoreValve. The value of capturing and perfecting this content in VR could completely change the dynamics of training courses, not only for this device but for all others as well.
Once the footage was obtained and processed by the Jaunt VR team I had the opportunity to experience it through Oculus technology. The results surpassed my expectations and therefore I stand by my initial vision. I firmly believe that with the right leadership and initiative this will be a game-changer in medicine. Why you may ask?
- Physicians learn procedures by repetition. The concept of ‘see one, do one, teach one’ is, in my opinion, is outdated. Maybe this worked in 1850 since it was the only option, but it is unacceptable in this day of age. VR allows the in-person demonstration to be repeated as many times as desired, and you can change your point-of-view with each viewing to take in even more information than if you were actually present for the procedure.
- The fact that one can be next to the individual performing the procedure over and over and focus on different aspects is priceless. Unlike traditional video displays, VR provides a human-scale, interactive facsimile that is potentially indistinguishable from reality.
- Exponential growth in technology drives down the price of many devices, including smartphones. By 2020 80% of the adult population will have a smartphone2. This is important because along with a $5 to $20 the Google Cardboard, your smartphone turns into a VR Headset3. Most recent android phones support Cardboard and you can try it for yourself with Jaunt VR’s publically available apps4. This does not mean that everyone should be producing content but it does mean that everyone can learn from it, from simple stitching to familiarizing oneself with more advanced technologies like CoreValve implantation.
Generating and sharing content has the potential to empower and educate thousands, maybe even millions, by experiencing procedures as the second operator or from any perspective that the operator deems to be best. As Dr. Oliver Aalami clearly put it “The reality is that it would take me one hour to review every step of a complex procedure that a video could show me in 10 minutes. In my opinion that is the true benefit…more immersive education…better comprehension in less time”
Technology is changing at an exponential rate. Just think of what merging VR technology with haptic feedback, ubiquitous networking, and tele-robotics can enable. Initially, education is the obvious application, but there is no reason why performing surgeries this way could not be done. Consider, for example, the successful telementoring project done back in 2003 between Brazil and the United States5.
I would like to emphasize that this would not have been possible without the support from Jaunt VR who allowed me to dream a little and get a taste of what I believe will be the beginning of a great thing for medical education.
In May 6, 2015 I will give a talk at SCAI regarding the incorporation of implementation of exponential thinking and technologies. Here I will share the immersive 360 experience with my Samsung Gear VR and other means as well.