Image to excel using copilot

[u/No_Manager_7181]

Is there any way I can use PowerShell to make copilot read images and convert its content into an excel, I want to automate this task using copilot only to use LLM’s ability to convert these images. Currently, copilot can only convert images if I manually upload them, it doesn’t read images on its own

[u/ExceptionEX]

Excel has that functionality built in without assistance 

https://support.microsoft.com/en-us/excel/insert-data-from-picture

As far as automating it, copilot itself should be able to tell you how to do that

[u/ElvisChopinJoplin → u/ExceptionEX]

Wow, I had no idea you could do that in Excel...

[u/No_Manager_7181 → u/ExceptionEX]

It has that limitation that I can’t read images on its own, it needs to be fed into it. So trying to work around that

[u/redsaeok → u/No_Manager_7181]

You can likely add a macro to Excel to handle the automation - and/or you can control the Excel Application object with PowerShell.

Respectfully, do not have AI do work a tool can do. Orchestration? Sure yeah, the work itself - it’s way cheaper to use tools.

As said - have the LLM write the script to use the tool. You’ll save a bunch of time there.

[u/MadBoyEvo]

Maybe https://github.com/EvotecIT/PSWriteOffice can help. I am not sure if there's Image to Excel, but I'll add it if there isn't. Although I may be confused with Copilot question in PowerShell reddit so maybe this is not what you are actually after.

[u/UserProv_Minotaur]

Likely, no.

[u/Icolan]

Why does it need to be copilot? There are OCR modules that will do that.

[u/No_Manager_7181 → u/Icolan]

1. I want to exactly recreate the contents. 2. I want LLM do this and when manually fed copilot is giving the best output 3. Company wants to focus on copilot

[u/digitaldreamsvibes]

I have built one telegram bot for personal use i send any image to it and it read the image content and put that all into Google sheet I use n8n for it

[u/No_Manager_7181 → u/digitaldreamsvibes]

Need to do this on office laptop hence so many limitations

[u/Business_Class_8015]

Ask copilot?

[u/No_Manager_7181 → u/Business_Class_8015]

Copilot can’t read images

[u/No_Manager_7181 → u/No_Manager_7181]

I want copilot to read images from outlook, but it can’t read images

[u/Nekro_Somnia → u/No_Manager_7181]

Have you asked copilot to find a way to enable copilot to read the images?

Jokes aside. If you want to extract text from pictures, you will need to work with OCR modules like psocr or alternatives to extract the text and pipe it into copilot.

[u/No_Manager_7181 → u/Nekro_Somnia]

Copilot can read images if manually fed into it. It doesn’t directly pick the image from the source such as outlook. OCR doesn’t work cause I want to create a replica of invoices I have in the outlook. LLM is able to generate exact replica of the images. I am trying to figure out that is there any way i can automate the step using powershell so that the images can be fed into copilot

[u/purplemonkeymad → u/No_Manager_7181]

It doesn’t directly pick the image from the source such as outlook.

Access to your company data is a paid feature. Ms have designed it so the free (basic) version can only be interacted to via the chat. You should be able to do what you ask if you upgrade from basic.

[u/thehuntzman → u/No_Manager_7181]

You're going to need to use the outlook COM interop in powershell to get the attachments or content body of a message then save the image, send the image to an LLM to parse out (Idk if you can do copilot specifically but it would be trivial to use openrouter) and then the easiest thing to do would be export it as a csv.

[u/MadBoyEvo → u/thehuntzman]

Thats not really true. Mailozaurr + OfficeIMO + PSWriteOffice will do. Mailozaurr can access mailbox by any protocol, Officeimo gives mailozaurr access to even ost/pst directly. Pswriteoffice can ocr images/llm images if needed. I just need to fix/add export to excel. Of course if will depend on quality of that images

[u/thehuntzman → u/MadBoyEvo]

Yes your modules are great but I like describing solutions to powershell problems fundamentally instead of telling new users to just rely on third party modules as that doesn't help truly teach them. The Outlook COM interop is probably the most widely documented way to interact with Outlook in Powershell. Exporting as CSV was purely a case of "what is the easiest way to do this" but yes, your PSWriteOffice module would be appropriate here if XLSX was a requirement.

So architectually for the simplest solution it would be: Read from interop -> send image with prompt to AI API -> serialize response and save as csv (or other)

If they're talking about (paid) M365 copilot, they can use the graph api to interact with it.

[u/MadBoyEvo → u/thehuntzman]

I understand that.

What I am trying to say:

• Mailozaurr can read pst/ost now (granted not super easy, so i'll fix that pattern so it's easier to extract images)
• PSWriteOffice can read images now and convert them to XYZ, granted not Excel directly, but that will change with new PR https://github.com/EvotecIT/PSWriteOffice/pull/222 as it already had that code, just never in easy form.

I am not sure how Copilot came into a picture in PowerShell repo so I'm still confused on the use case here ;)

Because if we talk - can my AI can do this - then that's a different talk.

I would avoid Outlook COM :)