虛擬模擬與人工智慧技術相結合將有可能為人文學科帶來一些創造性的應用，甚至會產生革命性的成果。透過量化比較研究人在真實環境和對應的虛擬模擬環境的心理和生理反應，我們能夠擺脫僅僅是感覺虛擬現實技術好像是有用的，但卻說不清哪裡有用的困境。或許此類研究能夠啟發我們將虛擬現實技術拓展到自己的研究領域...
Real vs. immersive-virtual emotional experience: Analysis of psycho-physiological patterns in a free exploration of an art museum
虛擬現實是人類行為研究中的強大工具，但是，很少有研究能夠與真實場景中引起相同情緒反應的能力進行對比。這項研究比較了自由參觀真實的藝術博物館和透過3D沉浸式虛擬環境（IVE）虛擬化的博物館體驗過程中引發的心理生理模式的特點；60名參與者參與此項探索性研究，使用可穿戴裝置記錄了腦電圖和心電圖訊號。使用自我評估情緒反應測試進行真實與虛擬心理的比較，透過支援向量機演算法進行生理比較，該演算法擅長特徵選擇程式，可用於量化心血管的一組最新指標以及大腦的線性和非線性動力學。在實驗的初始校準階段，使用標準化的2D和360°情感刺激來提高模型的準確性。物理和虛擬博物館的自我評估支援在情感研究中使用IVE。在物理博物館中，喚醒arousal和valece維度的二分類（高/低）系統精度分別為71.52％和77.08％，在虛擬博物館中，其分類準確度分別為75.00％和71.08％。研究結果表明大腦同步在情感虛擬現實過程中的高度參與。這些發現將有效地指導未來使用虛擬現實技術開發情緒啟發和識別系統。
Virtual reality is a powerful tool in human behaviour research. However, few studies compare its capacity to evoke the same emotional responses as in real scenarios. This study investigates psycho-physiological patterns evoked during the free exploration of an art museum and the museum virtualized through a 3D immersive virtual environment (IVE). An exploratory study involving 60 participants was performed, recording electroencephalographic and electrocardiographic signals using wearable devices. The real vs. virtual psychological comparison was performed using self-assessment emotional response tests, whereas the physiological comparison was performed through Support Vector Machine algorithms, endowed with an effective feature selection procedure for a set of state-of-the-art metrics quantifying cardiovascular and brain linear and nonlinear dynamics. We included an initial calibration phase, using standardized 2D and 360° emotional stimuli, to increase the accuracy of the model. The self-assessments of the physical and virtual museum support the use of IVEs in emotion research. The 2-class (high/low) system accuracy was 71.52% and 77.08% along the arousal and valence dimension, respectively, in the physical museum, and 75.00% and 71.08% in the virtual museum. The previously presented 360° stimuli contributed to increasing the accuracy in the virtual museum. Also, the real vs. virtual classifier accuracy was 95.27%, using only EEG mean phase coherency features, which demonstrates the high involvement of brain synchronization in emotional virtual reality processes. These findings provide an important contribution at a methodological level and to scientific knowledge, which will effectively guide future emotion elicitation and recognition systems using virtual reality.
目前，虛擬現實（VR）是模擬體驗並提供真實情況下的感覺方面最強大的技術之一。實際上，3D沉浸式虛擬環境（IVE）已成功應用於恐懼症，現場感，視覺化技術，體驗質量和影片遊戲。具體來說，該技術的主要優點是：i）它使我們在研究中能夠以有效且低成本的方式隔離和修改變數，這在真實環境中是非常困難，甚至是不可能的；ii）它使我們能夠在環境構建之前或遠離實驗室的環境中進行分析。值得注意的是，VR可以用來喚起情緒以及放鬆或焦慮的狀態。許多VR研究都是使用桌上型電腦或半沉浸式系統（例如Powerwalls或Caves）進行的。如今，由於虛擬現實頭盔（HMD）的效能提高和價格下降，其使用正在增加。它們是完全沉浸式的裝置，可將使用者與外界隔離。實際上，這些裝置激發了高度的存在感，可以理解為一種“存在”的錯覺。HMD具有兩種顯示IVE的形式：360°全景圖和3D VR環境。從參與者的心理反應來看，360°全景圖提供的結果更接近於現實，而從他們的生理反應來看，則是3D VR。此外，3D VR允許使用者與環境自由互動。
At present, Virtual Reality (VR) is one of the most powerful technologies that simulate experiences and provide the sensation of being in real situations [22]. In fact, 3D immersive virtual environments (IVE) have successfully been applied to phobias [23], presence [24], visualization technologies [25], quality of experience [26] and videogames [27]. Specifically, the main advantages of this technology are that: i) it allows us to isolate and modify variables in an efficient and low-cost way, something which is very difficult, or even impossible, in real environments [28]; and ii) it allows us to analyse an environment before its construction or environments far distant from the lab. Of note, VR can profitably be used to evoke emotions [28][29] and states of relaxation or anxiety [30]. Many VR researches have been performed using desktop or semi-immersive systems such as Powerwalls or caves [31]. Nowadays, the use of head-mounted displays (HMD) is growing due to their improved performance and decreased price. They are fully immersive devices that isolate the user from the external world. These devices, in fact, provoke a high sense of presence, understood as the illusion of "being-there" [32]. Note that HMDs have two main formats for displaying IVEs: 360° panoramas and 3D VR environments. 360° panoramas offer results closer to reality in terms of the participants' psychological responses, while 3D VR in terms of their physiological responses [33]. In addition, 3D VR allows the user to freely interact with the environment.
這項新穎的探索性研究的目的是透過3D IVE定量比較物理環境和虛擬環境中的情感體驗的心理和生理模式，從而指導未來使用VR的情感啟發和識別系統。考慮到這一目標，我們開發了一個現實美術館的3D IVE模擬，將對應真實美術館中的60個主題及其虛擬版本進行了比較研究。主要包括兩個先期階段，包括使用2D圖片和360°IVE進行受控刺激，以便研究此資料對情感模型的準確性和魯棒性的影響。在四個級別上討論了結果：i）比較心理測驗分數，ii）比較生理模式，iii）比較情緒識別水平和先前（標準化）控制刺激的影響，iv）比較了VR和真實場景中的情感主觀和心理生理相關性及其在不同的情感理論和模型框架中的含義。
The purpose of this novel and exploratory research was to quantitatively compare psychological and physiological patterns during an emotional experience in a physical environment and their virtualization through a 3D IVE, guiding future emotion elicitation and recognition systems using VR. With this aim in mind, we developed a realistic 3D IVE simulation of an art museum and performed a comparative study involving 60 subjects in a real art museum and its simulation, while they were performing a free exploration of an exhibition. In addition, we included two prior phases including controlled stimuli using 2D pictures and 360° IVEs, in order to study the influence of this data on the accuracy and robustness of the emotional models. The results can be discussed on four levels: i) a comparison of the psychometric scores, ii) a comparison of the physiological patterns, iii) a comparison of the level of emotion recognition and the influence of previously (standardized) controlled stimuli, iv) a comparison of emotional subjective and psychophysiological correlates in VR and real scenarios and its meaning in the framework of the different theories and models of emotion.
這一研究標誌著情感計算及其在環境生理學等領域的應用邁出了新的步伐，為情感體驗在真實環境和虛擬環境中的心理和生理比較提供了證據。本研究旨在克服傳統情感識別模型(如圖片、聲音、影片等)中被動方法的侷限性，為虛擬現實技術在情感提取中的應用提供支援。該方法在健康、建築設計、城市規劃和美學等許多學科的商業和研究層面都有影響。它可以應用於研究物件在許多特定環境下的情緒反應，如醫院、學校和工廠，那裡的使用者情緒反應在日常健康中很重要。更具體地說，新的情緒識別模型將有力地促進環境輔助生活的發展，這種智慧環境會根據人類的反應而變化。另一方面，新的VR設定允許分析一個引數的影響，改變它，同時保持環境的其餘部分在一個穩定的狀態。這將有助於發展許多由於物理原因(如空間的建築改造)或安全原因(如恐懼症治療)而無法在真實環境中進行的研究。此外，它將允許在建造前對環境進行分析，幫助決策過程中建立面向幸福的新環境。
This study marks new steps in the discipline of affective computing and its application to environmental physiology and other fields, providing evidence through psychological and physiological comparisons during an emotional experience in real and virtual environments. This exploratory study tries to contribute to overcome passive methods’ limitations of affective elicitation classically used in emotion recognition models, such as pictures, sounds or videos, supporting the use of VR in emotion elicitation. The methodology has implications at commercial and research levels in many disciplines as health, architectural design, urban planning and aesthetics. It could be applied to study the emotional responses of subjects in many specific environments, such as hospitals, schools and factories, where the emotional responses of users play a critical role in daily wellbeing. More specifically, new emotion recognition models will strongly contribute to the development of ambient assisted living, smart environments that change depending on human responses. On the other hand, the new VR set-up allows the analysis of the influence of one parameter, changing it while maintaining the remainder of the environment in a steady state. This will help to develop many studies, impossible to undertake in real environments for physical reasons (e.g. architectural modification of spaces) or security reasons (e.g. phobias therapy). Moreover, it will allow the analysis of environments before their construction, helping in the decision-making process of creating new environments oriented to wellbeing.
以上是論文[1]部分內容的翻譯，論文原文：https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0223881
[1] Marín-Morales J, Higuera-Trujillo JL, Greco A, Guixeres J, Llinares C, et al. (2019) Real vs. immersive-virtual emotional experience: Analysis of psycho-physiological patterns in a free exploration of an art museum. PLOS ONE 14(10): e0223881. https://doi.org/10.1371/journal.pone.0223881