_site/2024/04/11/edge_llm.html (105 lines of code) (raw):
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta
name="viewport"
content="width=device-width, initial-scale=1, shrink-to-fit=no"
/>
<meta name="description" content="Weekly AWS Networking Twitch Show" />
<meta name="author" content="" />
<meta property="og:title" content="Network Security Protections for LLMs" />
<meta property="og:description" content="Large Language Models are top of mind for many customers of AWS with the development of Gen AI applications. However, managing the applications applications against unwanted traffic can result in increased costs. In this Routing Loop Session, we will dive into the affects of bot traffic on LLM applications, the impact to costs, and how you can provide network layer security capabilities using AWS WAF and Amazon CloudFront to mitigate this traffic. At the end, we will dive into the different AWS chipsets that support these workloads, including AWS Trainium and AWS Inferentia." />
<meta property="og:image" content="https://www.theroutingloop.net/assets/image/link_background.jpg" />
<meta property="og:site_name" content="The Routing Loop" />
<title>The Routing Loop</title>
<!-- Bootstrap core CSS -->
<link href="/assets/css/main.css" rel="stylesheet" />
<script async src="https://www.googletagmanager.com/gtag/js?id=G-2K7T1C764L"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-2K7T1C764L');
</script>
</head>
<body>
<!-- Navigation -->
<nav class="navbar navbar-expand-lg navbar-dark bg-dark">
<div class="container">
<img src="/assets/image/520.AWS_AWS_logo_RGB_REV.png" alt="Logo" width=5% height="auto">
<button
class="navbar-toggler ml-auto"
type="button"
data-toggle="collapse"
data-target="#navbarResponsive"
aria-controls="navbarResponsive"
aria-expanded="false"
aria-label="Toggle navigation"
>
<span class="navbar-toggler-icon"></span>
</button>
<div class="collapse navbar-collapse" id="navbarResponsive">
<ul class="navbar-nav ml-auto">
<li class="nav-item">
<a class="nav-link" href="/">Home</a>
</li>
<li class="nav-item">
<a class="nav-link" href="/past/">Previous episodes</a>
</li>
<li class="nav-item">
<a class="nav-link" href="/upcoming/">Upcoming episodes</a>
</li>
<li class="nav-item">
<a class="nav-link" href="/hosts/">Hosts</a>
</li>
<li class="nav-item">
<a class="nav-link" href="https://pulse.aws/survey/6ONETCNV">Feedback</a>
</li>
</ul>
</div>
</div>
</nav>
<!-- Header -->
<header class="bg-primary py-5 mb-5" style="background-image: url('/assets/image/background.png');">
<div class="container h-100">
<div class="row h-100 align-items-center">
<div class="col-lg-12 text-center">
<h1 class="display-4 text-white mt-5 mb-2">
The Routing Loop
</h1>
<p class="lead mb-4 text-white-50">
<b>Wednesdays</b> 11 AM PT / 2 PM ET / 7 PM UK
</p>
<div class="d-flex justify-content-center">
<a href="https://www.twitch.tv/aws/" class="btn btn-light btn-lg mx-2" target="_blank">
<i class="fab fa-twitch"></i> Twitch
</a>
<!-- <a href="https://www.linkedin.com/company/amazon-web-services" class="btn btn-light btn-lg mx-2" target="_blank">
<i class="fab fa-linkedin"></i> LinkedIn
</a>
<a href="https://www.youtube.com/@AWSEventsChannel" class="btn btn-light btn-lg mx-2" target="_blank">
<i class="fab fa-youtube"></i> YouTube
</a>
<a href="https://www.facebook.com/amazonwebservices/ " class="btn btn-light btn-lg mx-2" target="_blank">
<i class="fab fa-youtube"></i> Facebook
</a>
--> </div>
</div>
</div>
</div>
</header>
<!-- Page Content -->
<div class="container mb-5">
<div class="content-area">
<span class="date">11 April 2024</span>
<h1>Network Security Protections for LLMs</h1>
<p><b>Hosts:</b><br />Riggs Goodman III</p>
<p><b>Guests:</b><br />Adam Boeglin, Principal SA, ML Infrastructure <br /> Alan Erdley, Principal Edge GTM Specialist <br /> EJ Chen, Sr Solutions Architect, Edge <br /> Justin Kurpius, Sr Edge Specialist</p>
<div class="abstract">
<b>Abstract:</b><br />Large Language Models are top of mind for many customers of AWS with the development of Gen AI applications. However, managing the applications applications against unwanted traffic can result in increased costs. In this Routing Loop Session, we will dive into the affects of bot traffic on LLM applications, the impact to costs, and how you can provide network layer security capabilities using AWS WAF and Amazon CloudFront to mitigate this traffic. At the end, we will dive into the different AWS chipsets that support these workloads, including AWS Trainium and AWS Inferentia.
</div>
<div class="video-container">
<iframe src="https://player.twitch.tv/?video=2117152694&parent=www.theroutingloop.net&parent=127.0.0.1&autoplay=false" height="315" width="560" allowfullscreen="" frameborder="0"></iframe>
</div>
<a href="https://pulse.aws/survey/6ONETCNV" class="button">Session Feedback/Content Suggestions</a>
</div>
</div>
<!-- /.container -->
<!-- Footer -->
<footer class="py-5 bg-dark">
<div class="container">
<p class="m-0 text-center text-white">
Copyright © 2025 Amazon Web Services, Inc. or its affiliates. All rights reserved
</p>
</div>
<!-- /.container -->
</footer>
<!-- Bootstrap core JavaScript -->
<script src="/assets/vendor/jquery/jquery.min.js"></script>
<script src="/assets/vendor/bootstrap/js/bootstrap.bundle.min.js"></script>
</body>
</html>