Raspberry Pi 4 VR Conference Call Assistant -- Episode 459

If I understand this set-up correctly i.e. you have a single physical conference room which has a robotic mannequin to represent each remote participant joining the call. Each participant in the call then looks at a particular mannequin as opposed to another participant's face.

Doesn't this then create a bit of a problem for conferencing applications where you typically would want to be able to see facial expressions or hand gestures of the person talking to you ? Or do you also get to see the remote participant's live webcam video alongside the mannequin webcam in the VR headset ? but then you would end up looking at another person wearing a VR headset which may be a bit weird.

One issue which may arise is the way that people look around a room. Sometimes you will turn your head and other times you will just move your eyes. Some form of eye-tracking may be required in addition to head tracking.

Hi Dave,

In a conference room, sitting around a table you won't really be able to just move your eyes to see everyone. You'd have to be sat quite far back!

Video of the other people would be awesome. If you all use the website and not the VR headset then you could see everyone and their facial expressions. I'd worry about home bandwidth there with sending and receiving video. But there's lots of future potential with this project!

Hi Lorraine.

Indeed, in real life head movement will vary with seating layout and distance. With VR it will also depend on the field of view of the cameras.

As you say, to see everyone in a room, you would have to move your head, but if you are just looking at the two or three people positioned directly opposite, then at times I suspect that you may only be moving your eyes between people as it takes less effort. If this is the case then in some instances you may see the mannequin move to look at you but other times not.

With the VR setup I suspect you could perhaps limit the horizontal field of view to cover just one person so that you always had to move your head to look at someone else, but that would end up giving you tunnel vision.

This will also likely depend on how you intend to place the mannequins. With the general move to replace physical space with virtual space these days then the conference room may end up having to be scaled down in size. Do you then place the mannequins optimised in a small circle in the corner of an office or do you stick with the more traditional layouts where you can never see the people to the extreme left or right of you ?

Also, how do you start to include presentation content. When your mannequin turns away to look at the presentation screen, then ideally your video feed needs to switch to the on-screen content instead.

(Mixed mode conferences are another issue that are cropping up where you may have 7 participants in the room and 70 remote, or alternatively 70 in the room and 7 remote.

Also there are now requests coming in for multi-site conferencing where you a) either stay at home and remote in b) drive to your nearest local centre to participate in a satellite group, c) travel to the main host venue.

Then beyond that there are the international conferences where the main conference venues are located across separate countries, each with country specific satellite venues and remote participants all wanting to communicate with each other as naturally as possible.)

I think the head tracking makes it a more natural user experience than the desktop virtual joystick control. However perhaps it needs more of a lighter weight smart glasses type solution to reduce the bulk of the VR headset. However the head tracking takes you away from the desktop device so everything has to be accessible through the VR.

(Looking back we had remote camera control on the old Tandberg video conference units back in the 90's. People got tired of it quickly though and ended up sticking with the one wide angle view for the duration.)

Network bandwidth has always been an issue with conferencing as it quickly adds up as the numbers of participants increase. As bandwidth increases, so does expectations of image quality. Minimising latency is another ongoing issue.

Have you looked at WebRTC which is gathering momentum ?

Hi Lorraine.

Indeed, in real life head movement will vary with seating layout and distance. With VR it will also depend on the field of view of the cameras.

As you say, to see everyone in a room, you would have to move your head, but if you are just looking at the two or three people positioned directly opposite, then at times I suspect that you may only be moving your eyes between people as it takes less effort. If this is the case then in some instances you may see the mannequin move to look at you but other times not.

With the VR setup I suspect you could perhaps limit the horizontal field of view to cover just one person so that you always had to move your head to look at someone else, but that would end up giving you tunnel vision.

This will also likely depend on how you intend to place the mannequins. With the general move to replace physical space with virtual space these days then the conference room may end up having to be scaled down in size. Do you then place the mannequins optimised in a small circle in the corner of an office or do you stick with the more traditional layouts where you can never see the people to the extreme left or right of you ?

Also, how do you start to include presentation content. When your mannequin turns away to look at the presentation screen, then ideally your video feed needs to switch to the on-screen content instead.

(Mixed mode conferences are another issue that are cropping up where you may have 7 participants in the room and 70 remote, or alternatively 70 in the room and 7 remote.

Also there are now requests coming in for multi-site conferencing where you a) either stay at home and remote in b) drive to your nearest local centre to participate in a satellite group, c) travel to the main host venue.

Then beyond that there are the international conferences where the main conference venues are located across separate countries, each with country specific satellite venues and remote participants all wanting to communicate with each other as naturally as possible.)

I think the head tracking makes it a more natural user experience than the desktop virtual joystick control. However perhaps it needs more of a lighter weight smart glasses type solution to reduce the bulk of the VR headset. However the head tracking takes you away from the desktop device so everything has to be accessible through the VR.

(Looking back we had remote camera control on the old Tandberg video conference units back in the 90's. People got tired of it quickly though and ended up sticking with the one wide angle view for the duration.)

Network bandwidth has always been an issue with conferencing as it quickly adds up as the numbers of participants increase. As bandwidth increases, so does expectations of image quality. Minimising latency is another ongoing issue.

Have you looked at WebRTC which is gathering momentum ?