Category : apache-flink

I have Windows 10 OS, docker-compose and want to work with Apache Flink tutorial Playground, docker-compose starting correctly starting docker-compose but after several minutes of work, Apache Flink has to create checkpoints, but there is some problem with access to the file system. Exception: org.apache.flink.runtime.checkpoint.CheckpointException: Could not finalize the pending checkpoint 104. Failure reason: Failure ..

Read more

How to control Flink’s jobs to be distributed/load-balanced(Evenly or another way where we can set the threshold limit for Free-Slots/Physical MEM/CPU Cores/JVM Heap Size etc..) properly amongst task-managers in a cluster? For example, I have 3 task-managers in a cluster where one task-manager is heavily loaded even though there are many Free Slots and other ..

Read more

I run a Kubernetes cluster, and I’m attempting to upgrade the flink clusters as they come with some long awaited features in the latest release 1.12 https://flink.apache.org/news/2020/12/10/release-1.12.0.html#kubernetes-high-availability-ha-service I take that the release is out already? When I attempt to update the image tag in the k8s deployment I only get "Back-off pulling image" messages. I ..

Read more

I’m running Flink on Kubernetes in a cluster of 10 nodes. I’m also using Minio for checkpointing/savepointg purposes. When I run Flink with my job.jar, I always get the following error: The program finished with the following exception: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Failed to execute job ‘StreamingTest’. at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:302) at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:198) at ..

Read more

I am trying to deploy a flink cluster with docker compose, this has been working fine, but whenever the job manager is restarted it drops all of the JAR files. According to the flink website: https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/docker.html I need to mount the folder location like this: – /host/path/to/job/artifacts:/opt/flink/usrlib however, upon successfully doing so, no jobs appear ..

Read more