Immersive Video Production / Project 1 / 360° Video Experimental Production

23.9.2024 - 14.10.2024 (Week 1 - Week 4)
Elysa Wee Qi En / 0355060 / Bachelor of Design (Hons) in Creative Media / Taylor's University
Immersive Video Production
Project 1 / 360° Video Experimental Production 

LECTURES 

Week 3 / 360 Video Storytelling [1]

• 360 videos are shot with omnidirectional lens that capture a full 360 view
• Both lenses can view around 200 degrees each rather than 180. There is some overlap to prevent an empty space or line.
• Both lenses used to record two separate videos that you'd have to stitch manually, but with newer cameras they do it automatically and create one video for you. 
• Viewers can choose where they look
• 360 videos are not 3D videos - the layers are clean and it is spherical. Imagine a balloon.
• VR and 360 videos are not quite the same 
• You can't move around in a 360 video unless the camera moves. In VR, you can also interact with the environment. 
• 360 videos are created with cameras recording the real world. VR is a digital simulated 3D universe that is created with a computer. 
• Wearing a headset is the best way to view (and experience) a 360 video 
• Degree of freedom (DOF):
• Yaw = looking left and right
• Pitch = looking up and down
• Roll = rolling the head (can't really do it in normal videos compared to 360 videos) 
• Can be used for live sports, concerts, documentaries, virtual travelling etc.
• Immersive experiences are when you get rid of the 'frame' of view

• The word 'panorama' was coined by a British painter, Robert Barker, in 1792

Fig. 1.1. Example of a panoramic shot

• In 1843, a camera with a 150 degree view was invented by Joseph Puchberger
• In 1857, a camera with a full 360 degree view was patented by M. Garrela of England
• In 1849, the stereoscope viewer was invented by Sir David Brewster
• Slight differences in the left and right views to create the illusion of 3D without being 3D
• The first VR headset was was created by Ivan Sutherland in 1968
• VR became used for military pilot training in 1979
• Due to high costs and danger of training by flying a real plane
• In 1984, VPL Research, one of the first companies to develop and sell VR products, was founded by Jaron Lanier
• VR gloves were created
• The first VR arcade machine was introduced in 1991
• In 2000, the first omnidirectional camera was invented

Communication is the ultimate objective of 360 storytelling.

• VR and 360 can give an immersive experience when viewing with a head-mounted display (HMD)
• Immersive is when you know it isn't real, but your brain still believes it is
• Immersion is being surrounded by images and sounds that create a realistic environment
• Presence = the feeling of actually existing in an environment = because you feel like it is real, you then feel like you are there
• But this mostly only happens when using a HMD
• You can have immersive experiences with books too, but you need to really focus on it and forget all your surroundings

• Storytelling, editing and storyboarding techniques, and cinematic shots used for traditional videos do not work for 360 videos. A typical stereo sound system doesn't work either.
• 360 videos are not meant to just 'tell' the viewer, but to allow them to 'experience'.
• Chosen ideas need to be taken from a different approach

Week 4 / Using the Insta 360 Pro 2

• Attach the antennas
• Use the '360 3D' mode on the app for VR stuff (stereoscopic)
• Otherwise can use '360 Pano' mode (monoscopic)
• Be careful to wait for it to officially start recording (stop loading) 
• Minimal editing can be done in the app, but in the laptop would be better as it can be quite heavy 
• Big file sizes
• Can use a USB hub with 7 SD Card readers, or use a LAN cable and change the camera's function to storage mode
• Don't record for too long and have breaks in between recordings to prevent it from heating up too much

INSTRUCTIONS

Fig. 2. Module Information Booklet

Project 1: 360° Video Experimental Production

For this first project, we were tasked with just coming up with a simple idea to film a 360 video about. It didn't have to be complicated or difficult to execute. We just had to basically experiment with the equipment and concept of a 360 video.

I managed to come up with two ideas at first:
1) Let the viewer feel like a tiny person doing parkour around a room or place. Using the camera itself as the 'person', I can try to make it do flips or jump from one object to another so the viewer can experience what it's like to be that small. 

2) Use the camera as a 'fork' just to see what a fork would see; from leaving the drawer, to being used etc. Could consider another object instead. 

However, I later learned that the camera would remain on a tripod during the filming process, so both these ideas are unusable. 

The reason I wanted to create something like this is because to me, a 360 video allows the viewer to experience something without physically being there or doing it. So, with that concept in mind, I came up with my final idea:

Let the viewer feel like someone who's participating in a photoshoot. Have the camera be stationed in front of the green screen, then have a photographer to take pictures and a makeup artist come in and fix their makeup.

This is the general outline of the scenes / shots / content that I first planned out:

Fig. 3.1. Outline of scenes [1] (28/9/2024)

But due to filming times, I decided to just cut out scene 4. It's mostly the same thing as scene 1 anyway, and would only be there to add more immersion. Not much experimentation with the 360 video format is involved or related to it.

So in the end, the scene outline is just this:

Fig. 3.2. Outline of scenes [2] (28/9/2024)

After recording and transferring the footage, we were guided to combine the clips into a video. I was able to use an effect to rotate the footage slightly as it was tilted, and also add some text as a title.

Fig. 3.3. Editing process (7/10/2024)

I added a fade in and out at the beginning and end of the video, and also a glow transition between clips. I also added title text, and some camera panning to follow the movement of the people in the video. This camera movement slowly revealed the title text as well. 

Fig. 3.4. Colour grading (7/10/2024)

Then I adjusted the colours of the footage so that both clips would have similar tones and lighting. 

FINAL 360° Video Experimentation Submission

Fig. 4. Final 360° Video Experimentation Submission (14/10/2024)

FEEDBACK

Week 4 
General Feedback:
- Don't make each scene in a 360 video too short, as it doesn't give the viewer time to immerse themself/feel immersed in the environment or look around and absorb/examine it 
- No need to shoot too many scenes
- First the viewer needs to figure out 'where they are' and 'why they are there', too short a time means they won't have time
- Establish point(s) of interest in a scene for the viewer to notice
- Don't let the scenes be random 
- Allow the viewer to be engaged with the video
- The theme doesn't matter as much as how you make it interesting, how you drag the audience into the video/experience 
- Don't do 3-act-storytelling in a 360 format, it doesn't work
- Try to recreate a dream you had (something illogical) 
Specific Feedback:
- Video is too long with not much going on
- Trimming it to 40 seconds would be better
- First scene can trim to 15 seconds
- Idea is alright but just too long
- Execution and intention is clear
- But not much room to revamp the concept for project 2
- Original first idea could be done in 360 with a set built, or if not then with a normal camera, or VR
- Try and overcome the roadblock of the first idea and come up with a creative solution
- Take some time to gather inspiration and let the idea come to you

REFLECTIONS

Experience
This first assignment was completed very early on in the semester. Since it was based on experimentation and ideation, our lecturer sent us home on the first week to think of our idea to film it on the second week. Filming was done quickly and easily, and then I was able to edit the footage simply and submit it. The feedback given was mostly after watching each others projects on a head-mounted device and discussing the effectiveness of our videos in conveying the feelings we intended. 

Observations
This project was my first experience filming a 360 video, and it quickly became clear that there were challenges to overcome. For example how a lot of techniques used in traditional 2D film are not applicable in filming 360 videos. This is most obvious in how 2D videos have camera angles that can be changed and adjusted, while 360 videos obviously show a full 360 view so it's not possible to 'hide things off screen' unless it is hidden behind something else or far enough away.

Findings
In Premiere Pro, there are tools specifically made for editing 360 videos. This is beneficial, but also all the other effects are not applicable. This limits the editing potential of 360 videos. However, the effects that do exist and are applicable can still be useful. For example, there is one effect that allows you to adjust the tilt of the footage if the camera had been tilted during filming, in order to sort of counterbalance it.