So I want to create bots that fight each other using tensorflow's object recognition. On each bot, there will be a camera attached, and I was curious if there is a way to retrieve the camera image?

In Godot 3.2 there is a new CameraServer class, which can be used to access the webcam/phone-camera/etc on a device. I don't know how it works myself, but that is likely how you can access the camera image.

I'd be cool to try IRL, but I have not that kind of money. I'm thinking of the in game camera. I think I can use a viewport node on each bot to do so. not sure tho.

Ah, I thought you were looking to get a real-world camera. My apologizes!

As for getting the image from a Godot camera, it depends on whether you are wanting to get images from multiple cameras or just a single camera (per Godot application).

To get the image from a single camera, you can use get_tree().root.get_texture().get_data() will return a static image. There is a demo on the Godot demo repository that might be helpful to look at. If you want an updating texture, which may or may not be compatible with TensorFlow, you can remove get_data() from the function to get a ViewportTexture (documentation), which will update as the Viewport renders.

If you want to get multiple camera textures, then you will need to have each Camera as a child of a Viewport node, where the Viewport node has the resolution you want (by default viewports are set to (0, 0), which will not produce an image). If you want both cameras to view the same scene, you need to make sure that own world is disabled, otherwise the camera will only see nodes that are children of the Viewport node. Then the code for accessing the image from each viewport can be achieved with the following (untested):

extends Spatial

var viewport_one
var viewport_two

func _ready():
	# Get the viewports
	viewport_one = get_node("path_to_viewport_one")
	viewport_two = get_node("path_to_viewport_two")
	
func _process(_delta):
	var image_one = viewport_one.get_texture().get_data()
	var image_two = viewport_two.get_texture().get_data()

Then from there you should have access to the both images from both cameras. From there you'll have to find a way to pass the image to TensorFlow, which I'm not sure how that would work.

3 years later