Evaluating the Robustness of Neural Networks: An Extreme Value. . . Experimental results on various networks, including ResNet, Inception-v3 and MobileNet, show that (i) CLEVER is aligned with the robustness indication measured by the $\ell_2$ and $\ell_\infty$ norms of adversarial examples from powerful attacks, and (ii) defended networks using defensive distillation or bounded ReLU indeed give better CLEVER
Counterfactual Debiasing for Fact Verification - OpenReview 016 namely CLEVER, which is augmentation-free 017 and mitigates biases on the inference stage 018 Specifically, we train a claim-evidence fusion 019 model and a claim-only model independently 020 Then, we obtain the final prediction via sub-021 tracting output of the claim-only model from 022 output of the claim-evidence fusion model,
Leaving the barn door open for Clever Hans: Simple features predict. . . This phenomenon, widely known in human and animal experiments, is often referred to as the ‘Clever Hans’ effect, where tasks are solved using spurious cues, often involving much simpler processes than those putatively assessed Previous research suggests that language models can exhibit this behaviour as well
TRANSFORMERS CAN NAVIGATE MAZES WITH MULTI-STEP PREDICTION - OpenReview the work identifies a Clever-Hans cheat based on shortcuts in teacher forced training similar to theo-retical shortcomings identified in Wang et al (2024b) This demonstrates that while transformers can represent world states for mazes, they may struggle in planning that requires significant foresight A
Submissions - OpenReview Leaving the barn door open for Clever Hans: Simple features predict LLM benchmark answers Lorenzo Pacchiardi , Marko Tesic , Lucy G Cheke , Jose Hernandez-Orallo 27 Sept 2024 (modified: 05 Feb 2025)
THE PITFALLS OF NEXT-TOKEN PREDICTION - OpenReview tokens We call this the Clever Hans cheat, named after a famous arithmetic-solving horse that was debunked to have been following subtle cues from a trainer Now, while the later tokens become easy to fit by the Clever Hans cheat, in contrast, the earlier answer tokens become impossible to learn This
LLaVA-OneVision: Easy Visual Task Transfer - OpenReview We present LLaVA-OneVision, a family of open large multimodal models (LMMs) developed by consolidating our insights into data, models, and visual representations in the LLaVA-NeXT blog series
Login - OpenReview Promoting openness in scientific communication and the peer-review process