PDF Balancing Performance and Efficiency: A Multimodal Large Language Model Pruning Method based Image Text Interaction

It’s a powerful LLM trained on a vast and diverse dataset, allowing it to understand various topics, languages, and dialects. GPT-4 has 1 trillion,not publicly confirmed by Open AI while GPT-3 has 175 billion parameters, allowing it to handle more complex tasks and generate more sophisticated responses. LLMs use a Chat GPT more complex architecture of neural networks called transformers, which differ from traditional neural networks in their ability to process entire sequences of data simultaneously rather than step-by-step. This allows transformers to capture long-range dependencies and contextual relationships more effectively.

For that reason, we do the same, but we evaluate the instruct-tuned base model as a point of comparison. We wish to enable rapid and low-cost MOE model creation to augment the capabilities of a given source model of interest. One needs only to select additional models with the same architecture as the source model as experts, and then combine the trained expert models with the source model of interest into an MOE. By selecting domain-specialised, trained models of interest to augment the capabilities of the source model, the resulting MOE model can deliver the promise of a true Mixture of Domain Experts. The first layer takes in a sequence of words as input, and each subsequent layer processes the output of the previous layer. The output of the last layer is the model’s prediction of the most likely meaning or interpretation of the input.

A finance-specific model will be able to improve existing financial NLP tasks, such as sentiment analysis, named entity recognition, news classification, and question answering, among others. However, we also expect that domain-specific models will unlock new opportunities. Many people have seen ChatGPT and other large language models, which are impressive new artificial intelligence technologies with tremendous capabilities for processing language and responding to people’s requests. However, we also need domain-specific models that understand the complexities and nuances of a particular domain. While ChatGPT is impressive for many uses, we need specialized models for medicine, science, and many other domains.

Recent LLMs have been used to build sentiment detectors,

toxicity classifiers, and generate image captions. This work was a collaboration between Bloomberg’s AI Engineering team and the ML Product and Research group in the company’s chief technology office, where I am a visiting researcher. This was an intensive effort, during which we regularly discussed data and model decisions, and conducted detailed evaluations of the model.

One of the lead engineers on this project is Shijie Wu, who received his doctorate from Johns Hopkins in 2021. Additionally, Gideon Mann, who received his PhD from Johns Hopkins in 2006, was the team leader. I think this shows the tremendous value of a Johns Hopkins education, where our graduates continue to push the scientific field forward long after graduation. Profit and prosper with the best of expert advice on investing, taxes, retirement, personal finance and more – straight to your e-mail.

Together we read all the papers we could find on this topic to gain insights from other groups, and we made frequent decisions together. Investors can use LLMs to explain complex investment strategies in simpler terms, ensuring they fully understand the rationale behind their financial decisions. LLMs can personalize investment strategies to fit individual investor needs. By analyzing an investor’s financial goals and risk tolerance, coupled with the current market conditions, LLMs can help investors create tailored portfolios that align with their objectives. This personalized approach ensures that each investment plan is unique to the investor’s specific circumstances, potentially leading to improved outcomes.

Looking ahead, I hope the conversation around AI and bias will continue to grow, incorporating more diverse perspectives and ideas. It requires us to stay committed to making AI more inclusive and representative of the diverse world we live in. By integrating this culturally informed feedback and comparison, I was able to make the AI-generated strategies more inclusive and culturally sensitive. In a 2023 study, researchers prompted four LLMs with a sentence that included a pronoun and two stereotypically gendered occupations. The LLMs were 6.8 times more likely to pick a stereotypically female job when presented with a female pronoun, and 3.4 times more likely to pick a stereotypically male job with a male pronoun4.

The strength of these connections, represented by weights, determines how much influence one neuron’s output has on another neuron’s input. During training, the network adjusts its weights based on examples from the dataset. A Large Language Model (LLM) is a deep learning algorithm large language models for finance that can recognise and interpret human language or other types of complex data. The “large” part of the name comes from LLMs training on massive data sets. Many LLMs are trained on data gathered from the Internet – thousands or millions of gigabytes’ worth of text.

What are some use cases for LLMs?

Meanwhile, a Chinese colleague noted that the AI failed to address the traditional use of herbal medicine and the importance of food therapy in Chinese culture. To ensure that bias doesn’t creep into my work when using LLMs, I adopt several strategies. First, I treat AI outputs as a starting point rather than as the final product.

The primary objective of trading is to forecast prices and generate profits based on these predictions. Initially, statistical machine learning methods such as Support Vector Machines (SVM) (jae Kim, 2003), Xgboost (Zolotareva, 2021), and tree-based algorithms were utilized for profit and loss estimation. Additionally, reinforcement learning (Wang et al., 2019) has been applied to automatic trading and portfolio optimization. We aim to leverage well-trained and effective expert modules and use them all as first-class citizens. We thus propose creating a Gate-less MOE, which assigns an equal weight to each expert.

Although these models are not as powerful as closed-source models like GPT-3 or PaLM(Chowdhery et al., 2022), they demonstrate similar or superior performance compared to similar-sized public models. Overall, BloombergGPT showcased commendable performance across a wide range of general generative tasks, positioning it favorably among models of comparable size. This indicates that the model’s enhanced capabilities in finance-related tasks do not come at the expense of its general abilities. It is important to note that the evolution of language models has mainly been driven by advancements in computational power, the availability of large-scale datasets, and the development of novel neural network architectures.

The key context, question, and desired answer are directly fed into the LLM, with the answer masked during training so that the model learns to generate it. Artificial Intelligence (AI) has witnessed extensive adoption across various domains of finance in recent years (Goodell et al., 2021). While this list is not exhaustive, these areas have shown significant interest and high potential with the advancement of AI. Our MOE Model Mixing toolkit swaps the FFN layers of each expert model, along with a gate, in place of the FFN layers of a base model. In [8], it was suggested to set the router parameters as the hidden state representations for each expert. We found that using this type of hidden representation in the gate does not work well.

Modeling human language at scale is a highly complex and resource-intensive

endeavor. The path to reaching the current capabilities of language models and

large language models has spanned several decades. In summary, this survey synthesized the latest progress in applying LLMs to transform financial AI and provided a practical roadmap for adoption. We hope it serves as a useful reference for researchers and professionals exploring the intersection of LLMs and finance. As datasets and computation improve, finance-specific LLMs represent an exciting path to democratize cutting-edge NLP across the industry. To provide adoption guidance, we proposed a structured framework for selecting the optimal LLM strategy based on constraints around data availability, compute resources, and performance needs.

Parameters

are the

weights

the model learned during training, used to predict the next token in the

sequence. “Large” can refer either to the number of parameters in the model, or

sometimes the number of words in the dataset. LLMs, like OpenAI’s GPT-4, are able to process and analyze complex information quickly, making them valuable tools in various industries, including finance. For investors, LLMs provide a means to sift through massive datasets, identify patterns, and generate insights that were previously difficult to obtain. Deep learning models can be used for supporting customer interactions with digital platforms, for client biometric identifications, for chatbots or other AI-based apps that improve user experience. Machine learning has also been often applied with success to the analysis of financial time-series for macroeconomic analysis1, or for stock exchange prediction, thanks to the large available stock exchange data.

They can also usually be repurposed for other tasks, a valuable silver lining. They can take months to train, and as a result

consume lots of resources. The self-attention mechanism determines the relevance of each nearby word to

the pronoun it. An

encoder converts input text into an intermediate representation, and a decoder

converts that intermediate representation into useful text.

For example, OpenAI ChatGPT cannot be used in areas that require confidentiality, such as state defence or healthcare. As the saying goes, you are what you eat, and in the case of generative AI, these programs process vast amounts of data and amplify the patterns present in that information. Language bias occurs because AI models are often trained on data sets dominated by English-language information. This often means that a model will perform better on an English-language task than it will on those in other languages, inadvertently sidelining people whose first language is not English. The Allen Institute for AI (AI2) developed the Open Language Model (OLMo).

How LLMs work

If the input question or context involves confidential data, it is necessary to proceed with the 1A action block, which involves self hosting an open-source LLM. As of July 2023, several options are available, including LLAMA(Touvron et al., 2023), OpenLLAMA(Geng and Liu, 2023), Alpaca(Taori et al., 2023), and Vicuna(Chiang et al., 2023). LLAMA offers models with sizes ranging from 7B to 65B, but they are limited to research purposes. OpenLLAMA provides options for 3B, 7B, and 13B models, with support for commercial usage. Alpaca and Vicuna are fine-tuned based on LLAMA, offering 7B and 13B options. Deploying your own LLM requires a robust local machine with a suitable GPU, such as NVIDIA-V100 for a 7B model or NVIDIA-A100, A6000 for a 13B model.

Investors must combine AI insights with their knowledge and judgment to make sound investment decisions. Ethical considerations around data privacy and responsible AI use should also be addressed. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. ArXiv is committed to these values and only works with partners that adhere to them.

The different sections are different evaluation tests with general knowledge and reasoning tests MMLU, MMLU-pro, ARC-challenge and GPT4All followed by two math test sets, GSM8K and GSM8K-COT. While the original Merlinite model performs well on MMLU and GPT4All, it is clear that its performance is lacking on the other evaluation tests. We aim therefore to complement Merlinite’s performance on other tasks without degrading its performance on MMLU and GPT4All. Focusing here on the first 2 grey bars and the first 2 yellow bars from the left, in each section, we see that Merlinite’s performance is maintained where it was good and is improved considerably where it was lacking.

Specifically, the author suggests compiling of a list of positive and negative prompts and then using a provided script which combines their hidden state representations by averaging and normalizing them, for each expert. Taking it one step further, we can imagine a set of trained models, each having a skill in a particular domain. Each MOE can mix a targeted subset of skill-based models to satisfy the distinct needs of each individual user.

This also accelerates computation by up to 2x since smaller data types speed up training. Moreover, the reduced memory footprint enables larger batch sizes, further boosting throughput. The choice as to which modules to mix into an MOE can be seen to be application and expert-model-dependent.

For instance (Radovanovic, 2023), Auto-GPT can optimize a portfolio with global equity ETFs and bond ETFs based on user-defined goals. It formulates detailed plans, including acquiring financial data, utilizing Python packages for Sharpe ratio optimization, and presenting the results to the user. Previously, achieving such end-to-end solutions with a single model was unfeasible. This property makes LLMs an ideal fit for financial customer service or financial advisory, where they can understand natural language instructions and assist customers by leveraging available tools and information.

Investors can use these technologies for in-depth market research and analysis, providing insights that inform better decision-making. LLMs can help optimize portfolios by suggesting asset allocations that maximize returns while minimizing risks. ArXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Check out the dedicated article the Speak Ai team put together on How Does Speech Recognition Work to learn more. Check out the dedicated article the Speak Ai team put together on The Best Executive Research Firms to learn more. You will get paid a percentage of all sales whether the customers you refer to pay for a plan, automatically transcribe media or leverage professional transcription services.

In practice, we should sample the token based on the probability distribution. Also, to make the tutorial concise, we execute the sample process on CPU. We load the pre-trained weights from Hugging Face and prepare the model weights. However, usually we only load the

pre-trained weight from Hugging Face but not the model architecture.

They are even beginning to show

strong performance on other tasks; for example, summarization, question

answering, and text classification. LLMs can even

solve some math problems and write code (though it’s advisable to check their

work). The experience of watching the model train over weeks is intense, as we examined multiple metrics of the model to best understand if the model training was working. Assembling the extensive evaluation and the paper itself was a massive team effort.

As shown in Table 2, there is a trend of combining public datasets with finance-specific datasets during the pretraining phase. Notably, BloombergGPT serves as an example where the corpus comprises an equal mix of general and finance-related text. It is worth mentioning that BloombergGPT primarily relies on a subset of 5 billion tokens that pertain exclusively to Bloomberg, representing only 0.7% of the total training corpus. This targeted corpus contributes to the performance improvements achieved in finance benchmarks. Language models created by big tech companies are aimed at the masses, and we have no control over them.

This iterative process of data preparation, model training, and fine-tuning ensures LLMs achieve high performance across various natural language processing tasks. They can be used in simple ways, see the worldwide success of Chat-GPT3, or fine-tuned to specific tasks. But it is more complex to redefine their architecture for new types of data, such as transactional bank data.

For the Noisy MOE with top-K𝐾Kitalic_K routing, an option of specifying an “always-on” expert is provided. Optional attention-layer mixing, along with the FFN mixing, is also supported. While we have found that the routers need not be trained to achieve good results, our toolkit offers the possibility to train the routers or a combination of the routers and the attention layers. In addition, while our results show that using the FFN layers of the experts is generally preferred, we also enable creating an MOE from LoRA adapter experts. They are trained on large datasets, such as the Common Crawl corpus and Wikipedia, to learn the structure and nuances of natural language. This allows them to generate new text in a similar style to the training data.

To better understand how these models work, let’s take a closer look at a step-by-step example. We trained a new model on this combined dataset and tested it across a range of language tasks on finance documents. Surprisingly, the model still performed on par on general-purpose benchmarks, even though we had aimed to build a domain-specific model. In collaboration with Bloomberg, we explored this question by building an English language model for the financial domain. We took a novel approach and built a massive dataset of financial-related text and combined it with an equally large dataset of general-purpose text. The resulting dataset was about 700 billion tokens, which is about 30 times the size of all the text in Wikipedia.

There are still limited guides and resources on how large language models work. Here is a video on “Large Language Models From Scratch” by Graphics in 5 Minutes. However, the advantage of using parts of words is that these can also appear in words the AI language model doesn’t know yet, making training more efficient. It does this by generating a probability distribution over the vocabulary for the next token. There is a large demand from our students to learn about how large language models work and how they can contribute to building them. In the past year alone, the Whiting School of Engineering’s Department of Computer Science has introduced three new courses that cover large language models to some degree.

• The output of the last layer is the model’s prediction of the most likely meaning or interpretation of the input.
• The output of each neuron is determined by its weights, which are adjusted as the model is trained.
• Our

  methodology provides a promising path to unlock LLMs’ potential for complex

  real-world domains.

• The project achieved preliminary results in the creation of a new foundation model for finances2, based on an evolution of the ‘Transformer’ architecture used by BERT, GPT and many other models.

These models have significantly enhanced language understanding and generation capabilities, enabling their application across a wide range of industries and domains. The key observation is that low-cost creation of an MOE from trained expert models is a viable approach to improving the performance of a model in a cost-effective manner. The first four bars from the left in each section of the figure show the evaluation results with the expert models individually.

Whenever I use generative AI to assist with research or writing, I always cross-check its results with trusted sources from various perspectives. It was a stark reminder of how important it is for AI systems to account for diversity. Strictly Necessary Cookie should be enabled at all times so that we can save your preferences for cookie settings. Tokenisation is the process of breaking down text into smaller units, often words or subwords. Training models with upwards of a trillion parameters

creates engineering challenges. Special infrastructure and programming

techniques are required to coordinate the flow to the chips and back again.

The finance industry could benefit from applying LLMs, as effective language understanding and generation can inform trading, risk modeling, customer service, and more. We provide a general-purpose toolkit for using trained models in a Mixture of Domain Experts MOE with a focus on the flexibility offered. While a router, or gate, can be trained on a small amount of relevant data to improve the performance of the Mixture of Domain Experts MOE, we find that it is not always necessary. Hence our toolkit offers the flexibility to create a Mixture of Domain Experts MOE in multiple ways including without router training. When there are only a few high-quality experts, our Gate-less MOE architecture can be the best solution. We find that the Gate-less architecture is competitive with and can even outperform router-based architectures yet is cheaper to produce.

A noisy gate can be used to reduce inference cost as compared to the Gate-free MOE, still not requiring any training, with generally only minor performance degradation. Both of these model mixing procedures allow for swapping in and out of expert models into an MOE at practically zero cost. We also offer the possibility to train the routers and examine the benefits that router training provides.

Bias can be a problem in very large models and should be considered in training

and deployment. If the input is “I am a good dog.”, a Transformer-based translator

transforms that input into the output “Je suis un bon chien.”, which is the

same sentence translated into French. The goal of the AI-X Foundry is to transform how Johns Hopkins conducts research through AI. Johns Hopkins researchers are among the world’s leaders in leveraging artificial intelligence to understand and improve the human condition. We recognize that a critical part of this goal is a strong collaboration between our faculty and industry leaders in AI, like Bloomberg. Building these relationships with the AI-X Foundry will ensure researchers have the ability to conduct truly transformative and cross-cutting AI research, while providing our students with the best possible AI education.

In addition to task-specific evaluations, general metrics used for LLMs can also be applied. Particularly, when evaluating the overall quality of an existing LLM or a fine-tuned one, comprehensive evaluation systems like the one presented in (Liang et al., 2022) can be utilized. This evaluation system covers tasks for various scenarios and incorporates metrics from different aspects, including accuracy, fairness, robustness, bias, and more. It can serve as a guide for selecting a language model or evaluating one’s own model in the context of finance applications. In standard fine-tuning, the model is trained on the raw datasets without modification.

In addition, our toolkit offers the capability to train the router or train a combination of the router and the embedding layers. You can foun additiona information about ai customer service and artificial intelligence and NLP. We also offer the possibility to create the MOE from trained LoRA adapters. Large language models are powerful tools used to process and analyze large amounts of text. They are based on deep learning algorithms and are trained on large datasets to learn the structure of natural language. Gemini is a multimodal LLM developed by Google and competes with others’ state-of-the-art performance in 30 out of 32 benchmarks. The Gemini family includes Ultra (175 billion parameters), Pro (50 billion parameters), and Nano (10 billion parameters) versions, catering various complex reasoning tasks to memory-constrained on-device use cases.

How do LLMs work?

It may also inform developers applying LLM solutions for the finance industry. Section 2 covers background on language modeling and recent advances leading to LLMs. Section 3 surveys current AI applications in finance and the potential for LLMs to advance in these areas. Sections 4 and 5 provide LLM solutions and decision guidance for financial applications.

After defining the model architecture, we can export the model to the Relax IRModule. After the AI created an initial draft of the prevention strategies, I shared the content with colleagues from each of these cultural backgrounds. She suggested including alternatives such as reducing portion sizes or incorporating low-glycemic-index rice varieties that align with Malay dietary practices.

Two major challenges are the production of disinformation and the manifestation of biases, such as racial, gender, and religious biases, in LLMs (Tamkin et al., 2021). In the financial industry, accuracy of information is crucial for making sound financial decisions, and fairness is a fundamental requirement for all financial services. To ensure information accuracy and mitigate hallucination, additional measures like retrieve-augmented generation (Lewis et al., 2021) can be implemented. To address biases, content censoring and output restriction techniques (such as only generating answers from a pre-defined list) can be employed to control the generated content and reduce bias. Training LLMs begins with gathering a diverse dataset from sources like books, articles, and websites, ensuring broad coverage of topics for better generalization. After preprocessing, an appropriate model like a transformer is chosen for its capability to process contextually longer texts.

We show that when the number of expert models is small, this can be an optimal strategy for MOE model mixing in terms both of model creation cost and subsequent model performance on evaluation tasks. However, when the number of expert models grows, one may wish to avail of the sparsity that a top-K𝐾Kitalic_K strategy affords, whereby only the top K𝐾Kitalic_K expert modules are activated for each token. We show in Section 4 that this Noisy MOE works almost as well as the Gate-less MOE and provides faster inference time when there are more than 2 experts. Large language models are also used to identify the sentiment of text, such as in sentiment analysis. They can be used to classify documents into categories, such as in text classification tasks. They are also used in question-answering systems, such as in customer service applications.

However, their results are limited to comparing perplexity across the weighting strategies. In addition, their scoring methods are not practical, as they require in general each adapter’s “training domain dataset” to evaluate proximity to the input. The authors of [12] propose a similar approach but require the experts to be LoRA adapters and the use of a single linear-layer router shared across all of the LoRA layers. We note that a LoRA-adapter based architecture can be achieved with our methods and toolkit, along with further flexibility that we provide in the definition of the experts and the routing mechanism. They are used to generate natural-sounding text, such as in chatbots and virtual assistants.

AI-powered chatbots, as discussed in (Misischia et al., 2022), already provide more than 37% of supporting functions in various e-commerce and e-service scenarios. In the financial industry, chatbots are being adopted as cost-effective alternatives to human customer service, as highlighted in the report ”Chatbots in consumer finance” (Cha, 2023). Additionally, banks like JPMorgan are leveraging AI services to provide investment advice, as mentioned in a report by CNBC (Son, 2023).

We create a

financial LLM (FLLM) using multitask prompt-based finetuning to achieve data

pre-processing and pre-understanding. To overcome manual annotation costs, we employ abductive augmentation

reasoning (AAR) to automatically generate training data by modifying the pseudo

labels from FLLM’s own outputs. Experiments show our data-centric FLLM with AAR

substantially outperforms baseline financial LLMs designed for raw text,

achieving state-of-the-art on financial analysis and interpretation tasks. We

also open source a new benchmark for financial analysis and interpretation.

The authors of [11] assume that users of a mixed MOE will fully fine-tune the resulting MOE. We suspect that fine-tuning the mixed MOE for a few epochs results in the MOE losing the ability of its expert modules to handle well the domains that each expert was trained on. Large language models are powerful tools used by researchers, companies, and organizations to process and analyze large volumes of text. These models are capable of understanding natural language and can be used to identify meanings, relationships, and patterns in text-based data. In recent years, the field of Natural Language Processing (NLP) has witnessed a remarkable surge in the development of large language models (LLMs). Due to advancements in deep learning and breakthroughs in transformers, LLMs have transformed many NLP applications, including chatbots and content creation.

The output of each neuron is determined by its weights, which are adjusted as the model is trained. Building these models isn’t easy, and there are a tremendous number of details you need to get right to make them work. We learned a lot from reading papers from other research groups who built language models. We also released detailed “training chronicles” that contains a narrative description of the model-training process. Our goal is to be as open as possible about how we built the model to support other research groups who may be seeking to build their own models. The first decision block determines whether to use an existing LLM service or an open-source model.

AIML – Sr. Machine Learning Engineer – Large Language Models and Generative AI, Siri Information and Intelligence

These model variants follow a pay-per-use policy but are very powerful compared to others. Let’s explore these top 8 language models influencing NLP in 2024 one by one. This embedding is a high-dimensional vector that represents the token in a continuous vector space, capturing the semantic and syntactic meanings, often within a https://chat.openai.com/ specific context. As these models are trained on human language, this can introduce numerous

potential ethical issues, including the misuse of language, and bias in race,

gender, religion, and more. LLMs are highly effective at the task they were built for, which is generating

the most plausible text in response to an input.

Compared to other supervised models, LLMs offer superior adaptation and flexibility. Instead of training separate models for specific tasks, LLMs can handle multiple tasks by simply modifying the prompt under different task instructions (Brown et al., 2020b). This adaptability does not require additional training, enabling LLMs to simultaneously perform sentiment analysis, summarization, and keyword extraction on financial documents. We examine several variants of the methodology on a different model, llama3-8B.

The Noisy MOE evaluation result is shown as a red horizontal line in each plot on the right, while the best evaluation result of the expert models in each MOE is shown as a red dashed line. In [13] the authors propose an “on-demand selection and combination” of LoRA adapters at inference time and provide a their code publicly. Their method consists of a scoring strategy to identify the top K𝐾Kitalic_K adapters and various weighting (and parameter averaging and ensembling) strategies for combining the adapters.

A transformer has multiple layers of self-attention mechanisms and feed-forward neural networks. The self-attention mechanism helps the model focus on different parts of the input sentence to understand the context. These weighted connections link neurons in adjacent layers, which transmit signals from one layer to the next.

The framework aims to balance value and investment by guiding practitioners from low-cost experimentation to rigorous customization. We train the routers of the 2X MOE and the 4X MOE and examine the loss curves from the training. We also compare the results of the evaluation tests across these training paradigms. While the loss appears to decrease more when instruct-tuning both routers and embedding layers, the results are not borne out in evaluation, shown on the right in Figures 3 and 4.

Technically, it belongs to a class of small language models (SLMs), but its reasoning and language understanding capabilities outperform Mistral 7B, Llamas 2, and Gemini Nano 2 on various LLM benchmarks. However, because of its small size, Phi-2 can generate inaccurate code and contain societal biases. As models are built bigger and bigger, their complexity and efficacy increases.

• These advanced AI tools are changing the way investment strategies are developed and implemented, offering unprecedented opportunities for investors.
• By integrating this culturally informed feedback and comparison, I was able to make the AI-generated strategies more inclusive and culturally sensitive.
• We also offer the possibility to train the routers and examine the benefits that router training provides.

By formulating explicit instructions and demonstrations in the training data, the model can be optimized to excel at certain tasks or produce more contextually relevant and desired outputs. The instructions act as a form of supervision to shape the model’s behavior. The current implementation of deep learning models offers significant advantages by efficiently extracting valuable insights from vast amounts of data within short time frames. This capability is particularly valuable in the finance industry, where timely and accurate information plays a crucial role in decision-making processes. With the emergence of LLMs, even more tasks that were previously considered intractable become possible, further expanding the potential applications of AI in the finance industry. Financial text mining represents a popular area where deep learning models and natural language processing techniques are extensively utilized.