Early release of the official implementation for "GraphMETRO: Mitigating Complex Graph Distribution Shifts via Mixture of Aligned Experts"
-
Updated
May 22, 2024 - Python
Early release of the official implementation for "GraphMETRO: Mitigating Complex Graph Distribution Shifts via Mixture of Aligned Experts"
Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization
Analysis of token routing for different implementations of Mixture of Experts
This instruction aims to reproduce the results in the paper “Mesh-clustered Gaussian process emulator for partial differential equation boundary value problems”(2024) to appear in Technometrics.
Using CCR to predict piezoresponse force microscopy datasets
Gaussian Process-Gated Hierarchical Mixture of Experts
Anomaly Detection by Recombining Gated Unsupervised Experts
This is a prototype of a MixtureOfExpert LLM made with pytorch. Currently in developpment, I am testing its capabilities of learning with simple little tests before learning it on large language datasets.
About Code repository for: Nguyen, H., Nguyen, T., Nguyen, K., & Ho, N. (2024). Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts. In Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, AISTATS 2024, Acceptance rate 27.6% over 1980 submissions.
MoE Decoder Transformer implementation with MLX
The implementation of mixtures for different tasks.
Code, data, and pre-trained models for our EMNLP 2021 paper "Think about it! Improving defeasible reasoning by first modeling the question scenario"
Faster alternative to Fast Feedforward Layer that uses angular distance for routing
[Preprint] Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Experts
[Paper][Preprint 2024] Mixture of Modality Knowledge Experts for Robust Multi-modal Knowledge Graph Completion
This is the repo for the MixKABRN Neural Network (Mixture of Kolmogorov-Arnold Bit Retentive Networks), and an attempt at first adapting it for training on text, and later adjust it for other modalities.
Differentially private retriever using transformer memory as a search index for information retrieval
Add a description, image, and links to the mixture-of-experts topic page so that developers can more easily learn about it.
To associate your repository with the mixture-of-experts topic, visit your repo's landing page and select "manage topics."