✅博主简介：擅长数据搜集与处理、建模仿真、程序设计、仿真代码、论文写作与指导，毕业论文、期刊论文经验交流。

✅成品或者定制，扫描文章底部微信二维码。

(1) 基于蒙特卡洛丢弃法的认知不确定性建模方法

微表情识别任务面临数据样本稀缺的挑战,现有的公开数据集规模有限,如果仅以提升模型准确率为训练目标,模型容易出现过拟合现象,在测试数据上的泛化能力较差。本研究引入贝叶斯深度学习框架中的认知不确定性对模型的不确定性进行建模,认知不确定性源于模型参数的不确定性,反映了由于训练数据不足导致模型对其学习到的知识缺乏信心的程度。在贝叶斯神经网络中,模型参数被视为随机变量而非固定值,通过对参数的后验分布进行推断来量化模型的不确定性。然而精确计算参数后验分布在深度神经网络中是不可行的,本研究采用蒙特卡洛丢弃法对后验分布进行近似推断。具体而言,在网络的全连接层和卷积层后添加丢弃层,在测试阶段保持丢弃层激活,对同一输入进行多次前向传播,每次传播随机丢弃不同的神经元连接,相当于从参数后验分布中采样得到不同的网络实例。将多次前向传播结果的均值作为最终预测,方差作为认知不确定性的度量。当模型对某个样本的预测具有较大方差时,表明模型对该样本的分类结果不够确信,可能需要更多的训练数据来减少这种不确定性。实验结果表明,通过对认知不确定性进行建模,模型能够识别出那些难以分类的边界样本,并在这些样本上给出较低的置信度,提高了预测结果的可靠性。

(2) 基于损失函数衰减的任意不确定性表征方法

微表情数据的采集过程中不可避免地存在测量误差、标注错误等噪声,这些数据固有的不确定性会影响模型的训练效果和预测精度。本研究采用贝叶斯深度学习中的任意不确定性来捕获数据固有的噪声,任意不确定性与输入数据相关,是输入特征的函数,即使有无限多的训练数据也无法消除这种不确定性。在技术实现上,本研究对网络的损失函数进行改进,使网络能够同时预测分类结果和对应的任意不确定性。网络的输出层包含两个分支,一个分支输出各类别的预测概率,另一个分支输出与输入样本相关的不确定性估计。损失函数被设计为负对数似然形式,将任意不确定性作为方差参数引入,当某个样本的预测不确定性较大时,该样本对损失函数的贡献会被相应衰减,从而减少噪声数据对模型训练的负面影响。这种设计使模型能够自适应地降低噪声样本的权重,将注意力集中在高质量的训练样本上。任意不确定性的引入还提供了一种数据质量评估机制,那些被模型赋予较高任意不确定性的样本可能存在标注错误或采集噪声,值得进一步人工核查。实验表明,通过对任意不确定性进行建模,模型对噪声数据的鲁棒性显著提升,在含有标注噪声的数据集上表现更加稳定。

(3) 基于光流特征的三流卷积神经网络微表情识别框架

本研究设计了一套完整的微表情识别流程,首先对微表情视频序列进行预处理,定位微表情的顶点帧。顶点帧是微表情强度最大的帧,包含最丰富的表情特征信息。对于未标注顶点帧的数据集,采用基于光流幅值的自动定位算法,计算相邻帧之间的光流场,选择光流幅值最大的帧作为顶点帧。定位到顶点帧后,计算顶点帧与起始帧之间的光流特征,包括水平方向光流、垂直方向光流和光学应变三个分量。水平和垂直光流描述了面部各区域的运动方向和速度,光学应变反映了面部肌肉的形变程度,三者相互补充,能够全面刻画微表情的动态变化模式。将三种光流特征作为三个通道输入到三流三维卷积神经网络中,网络的三个分支分别处理三种光流特征,通过三维卷积核同时提取空间和时间维度的特征,捕获微表情的时空演变规律。三个分支的特征在全连接层进行融合,最终输出微表情的类别预测和不确定性估计。

import numpy as np import torch import torch.nn as nn import torch.nn.functional as F import torch.optim as optim from torch.utils.data import DataLoader, TensorDataset import cv2 class OpticalFlowExtractor: def __init__(self): self.farneback_params = dict(pyr_scale=0.5, levels=3, winsize=15, iterations=3, poly_n=5, poly_sigma=1.2, flags=0) def compute_flow(self, frame1, frame2): gray1 = cv2.cvtColor(frame1, cv2.COLOR_BGR2GRAY) gray2 = cv2.cvtColor(frame2, cv2.COLOR_BGR2GRAY) flow = cv2.calcOpticalFlowFarneback(gray1, gray2, None, **self.farneback_params) return flow def compute_optical_strain(self, flow): fx = cv2.Sobel(flow[:, :, 0], cv2.CV_64F, 1, 0, ksize=3) fy = cv2.Sobel(flow[:, :, 1], cv2.CV_64F, 0, 1, ksize=3) strain = np.sqrt(fx ** 2 + fy ** 2) return strain def extract_features(self, onset_frame, apex_frame): flow = self.compute_flow(onset_frame, apex_frame) horizontal_flow = flow[:, :, 0] vertical_flow = flow[:, :, 1] optical_strain = self.compute_optical_strain(flow) features = np.stack([horizontal_flow, vertical_flow, optical_strain], axis=0) return features class ApexFrameDetector: def __init__(self): self.flow_extractor = OpticalFlowExtractor() def detect_apex(self, frames): max_magnitude = 0 apex_index = 0 onset_frame = frames[0] for i in range(1, len(frames)): flow = self.flow_extractor.compute_flow(onset_frame, frames[i]) magnitude = np.mean(np.sqrt(flow[:, :, 0] ** 2 + flow[:, :, 1] ** 2)) if magnitude > max_magnitude: max_magnitude = magnitude apex_index = i return apex_index class ThreeStream3DCNN(nn.Module): def __init__(self, num_classes, dropout_rate=0.5): super(ThreeStream3DCNN, self).__init__() self.stream1 = self._create_stream() self.stream2 = self._create_stream() self.stream3 = self._create_stream() self.fusion_fc = nn.Linear(256 * 3, 512) self.dropout = nn.Dropout(dropout_rate) self.classifier = nn.Linear(512, num_classes) self.uncertainty_head = nn.Linear(512, num_classes) def _create_stream(self): return nn.Sequential( nn.Conv3d(1, 32, kernel_size=(3, 3, 3), padding=1), nn.BatchNorm3d(32), nn.ReLU(), nn.MaxPool3d(kernel_size=(1, 2, 2)), nn.Conv3d(32, 64, kernel_size=(3, 3, 3), padding=1), nn.BatchNorm3d(64), nn.ReLU(), nn.MaxPool3d(kernel_size=(1, 2, 2)), nn.Conv3d(64, 128, kernel_size=(3, 3, 3), padding=1), nn.BatchNorm3d(128), nn.ReLU(), nn.AdaptiveAvgPool3d((1, 4, 4)), nn.Flatten(), nn.Linear(128 * 16, 256), nn.ReLU() ) def forward(self, x1, x2, x3): f1 = self.stream1(x1) f2 = self.stream2(x2) f3 = self.stream3(x3) fused = torch.cat([f1, f2, f3], dim=1) fused = F.relu(self.fusion_fc(fused)) fused = self.dropout(fused) logits = self.classifier(fused) log_var = self.uncertainty_head(fused) return logits, log_var class BayesianMicroExpressionModel(nn.Module): def __init__(self, num_classes, mc_dropout_rate=0.3): super(BayesianMicroExpressionModel, self).__init__() self.base_model = ThreeStream3DCNN(num_classes, dropout_rate=mc_dropout_rate) self.mc_dropout_rate = mc_dropout_rate self.num_mc_samples = 20 def mc_forward(self, x1, x2, x3): self.train() predictions = [] uncertainties = [] for _ in range(self.num_mc_samples): logits, log_var = self.base_model(x1, x2, x3) predictions.append(F.softmax(logits, dim=1)) uncertainties.append(log_var) predictions = torch.stack(predictions, dim=0) mean_prediction = predictions.mean(dim=0) epistemic_uncertainty = predictions.var(dim=0).sum(dim=1) aleatoric_uncertainty = torch.stack(uncertainties, dim=0).mean(dim=0) return mean_prediction, epistemic_uncertainty, aleatoric_uncertainty def forward(self, x1, x2, x3, return_uncertainty=False): if return_uncertainty: return self.mc_forward(x1, x2, x3) else: return self.base_model(x1, x2, x3) class HeteroscedasticLoss(nn.Module): def __init__(self): super(HeteroscedasticLoss, self).__init__() def forward(self, logits, log_var, targets): precision = torch.exp(-log_var) ce_loss = F.cross_entropy(logits, targets, reduction='none') weighted_loss = precision * ce_loss + log_var return weighted_loss.mean() def train_bayesian_model(model, train_loader, val_loader, epochs, learning_rate): optimizer = optim.Adam(model.parameters(), lr=learning_rate, weight_decay=1e-4) criterion = HeteroscedasticLoss() scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode='min', patience=5, factor=0.5) for epoch in range(epochs): model.train() train_loss = 0 for batch in train_loader: x1, x2, x3, labels = batch optimizer.zero_grad() logits, log_var = model(x1, x2, x3) loss = criterion(logits, log_var, labels) loss.backward() optimizer.step() train_loss += loss.item() model.eval() val_loss = 0 correct = 0 total = 0 with torch.no_grad(): for batch in val_loader: x1, x2, x3, labels = batch mean_pred, epistemic, aleatoric = model(x1, x2, x3, return_uncertainty=True) predicted = mean_pred.argmax(dim=1) correct += (predicted == labels).sum().item() total += labels.size(0) scheduler.step(val_loss) return model def calculate_uf1_uar(predictions, targets, num_classes): class_recalls = [] class_f1s = [] for c in range(num_classes): tp = ((predictions == c) & (targets == c)).sum() fn = ((predictions != c) & (targets == c)).sum() fp = ((predictions == c) & (targets != c)).sum() recall = tp / (tp + fn + 1e-8) precision = tp / (tp + fp + 1e-8) f1 = 2 * precision * recall / (precision + recall + 1e-8) class_recalls.append(recall) class_f1s.append(f1) uar = np.mean(class_recalls) uf1 = np.mean(class_f1s) return uf1, uar if __name__ == "__main__": num_classes = 3 model = BayesianMicroExpressionModel(num_classes=num_classes, mc_dropout_rate=0.3) flow_extractor = OpticalFlowExtractor() apex_detector = ApexFrameDetector() batch_size = 16 dummy_x1 = torch.randn(batch_size, 1, 8, 64, 64) dummy_x2 = torch.randn(batch_size, 1, 8, 64, 64) dummy_x3 = torch.randn(batch_size, 1, 8, 64, 64) dummy_labels = torch.randint(0, num_classes, (batch_size,)) train_dataset = TensorDataset(dummy_x1, dummy_x2, dummy_x3, dummy_labels) train_loader = DataLoader(train_dataset, batch_size=8, shuffle=True) val_loader = DataLoader(train_dataset, batch_size=8, shuffle=False)

如有问题，可以直接沟通

👇👇👇👇👇👇👇👇👇👇👇👇👇👇👇👇👇👇👇👇👇👇