Test on Generative Models: LLMs and Transformers

Question 1: Neural network generative models are utilized for the de novo design of protein structure and function.

A. Ano

B. Ne

Explanation: The study materials explicitly state 'Protein Generated with RFDiffusion' as an application, with the note 'De novo design of protein structure and function with RFdiffusion'.

Question 2: According to the study materials, for what primary purpose are transformer models increasingly used in modern language modeling?

A. Predicting the next word based on previous words.

B. Identifying protein structures in biological data.

C. Generating high-resolution images.

D. Creating new chemical reaction predictions.

Explanation: The study materials state that the basic idea of large language models is next 'word' prediction, and transformer models are discussed as a type of neural network architecture used in modern language models for tasks like generation. The other options, while mentioned as applications of generative models, are not specifically attributed to transformer models in the context of language modeling within these materials.

Question 3: The position embedding matrix provides a vectorial encoding of the word positions.

A. Ano

B. Ne

Explanation: The study materials explicitly state: 'The position embedding matrix 𝑊 + provides a vectorial encoding of the word positions'.

Question 4: According to the study materials, what type of graph does the attention mechanism learn to predict?

A. A weighted, directed graph

B. An unweighted, undirected graph

C. A simple, unweighted graph where all nodes are identical

D. A graph exclusively used for comparing words across different languages

Explanation: The study materials explicitly state that the attention mechanism learns to predict a 'weighted, directed graph' or a 'weighted, directed (bipartite) graph'.

Question 5: The outputs from the multiple attention heads are summed together to create a single output before being fed into subsequent normalization and feedforward layers.

A. Ano

B. Ne

Explanation: The study materials explicitly state that the outputs from the multiple attention heads are concatenated to get a single output, not summed, before a final linear transformation and being fed into normalization and feedforward layers.