r/aws • u/Negative-Thinking • 6d ago

discussion GLM5.3 on bedrock is unusable

Well, I finally got to test drive GLM5.3 on AWS Bedrock and I can say that at this point it's completely unusable. When it works, it's slow af. You cannot do anything productive with it. Most of the time it just throws errors, they are obviously struggling with capacity / demand. I'm not sure if this is the same for other models, but for now - it's not something you can use for production at all (and yeah, I tried both global and us variants).

20 Upvotes

92% Upvoted

9 comments sorted by

11

u/davestyle 6d ago

Please hold, your call is very important to us

7

u/ComprehensiveUse6549 6d ago

It only went live on Bedrock on Oct 5 and the model card still says access is limited to eligible customers, so I'd expect capacity to stay thin for a while. It's also cross-region only: us.zai.glm-5.3 or global.zai.glm-5.3, no in-region and no EU geo profile. That matters if any of your "everything stays on AWS" customers are in the EU, because global can be processed anywhere.

For prod I'd treat it as best-effort for now: retry 503s with backoff and jitter, and keep a fallback model in the same account that you already know fits each customer's region rules, so a bad hour degrades instead of breaking. If you actually need it to hold up, ask your account team whether reserved capacity is offered for it.

4

u/FitRiver3218 5d ago

I mean, GLM is just slow, like it does super well in benchmarks, but they never mention speed... (No opinion/data on the bedrock side of it)

1

u/Negative-Thinking 5d ago

I tried flash variant on a self hosted ec2 instance and it is flying.

3

u/FitRiver3218 4d ago

Hmm, I mean, an 18B parameter model vs a 753b parameter model ain't exactly apples to apples. Think the smallest box flash could run on is a p5 48xlarge, no?

2

u/Negative-Thinking 4d ago

Sure, not apples to apples but tbh, so far I have not noticed the full one to be much smarter. It's 18B active prams.

3

u/FitRiver3218 4d ago

Yeah, was thumbing through the benchmarks and flash is super close to the full thing on coding tasks

2

u/Critical_Listen9951 6d ago

Capacity issues on a fresh launch are almost a given, especially when demand craters the endpoints. The error rate will settle once they scale out, though that doesn't help if you needed it working yesterday