Wednesday, July 10, 2024
HomeiOS Developmentios - Picture recognition CoreMl + SwiftUi.... Difficulty with view

ios – Picture recognition CoreMl + SwiftUi…. Difficulty with view


I am experiencing a problem with my SwiftUI undertaking that makes use of Firebase and a CoreML mannequin for picture recognition. Regardless of including numerous debug print statements, the method would not proceed past the validation step. Beneath are the main points of my setup and the related code:

Downside Description
Once I try and create a brand new submit, the console logs “Submit button tapped” and “Fields validated”, however nothing occurs afterward. The submit just isn’t created, and there aren’t any additional console logs to point what went flawed. Moreover, I am getting a number of CoreGraphics and constraint-related warnings within the console.

Console Output

Mannequin loaded efficiently
Loading posts...
Person doc doesn't exist or no following information
Loaded 8 fallback posts
> > <0x13de085e0> Gesture: System gesture gate timed out.
> > <0x13de085e0> Gesture: System gesture gate timed out.
Error: this utility, or a library it makes use of, has handed an invalid numeric worth (NaN, or not-a-number) to CoreGraphics API and this worth is being ignored. Please repair this downside.
If you wish to see the backtrace, please set CG_NUMERICS_SHOW_BACKTRACE environmental variable.
Error: this utility, or a library it makes use of, has handed an invalid numeric worth (NaN, or not-a-number) to CoreGraphics API and this worth is being ignored. Please repair this downside.
If you wish to see the backtrace, please set CG_NUMERICS_SHOW_BACKTRACE environmental variable.
Submit button tapped
Fields validated

`NewPostAlertState.swift`
import Basis
import SwiftUI

enum NewPostAlertState: Identifiable {
    case error(String)
    case success(String)
    case affirmation(() -> Void)
    
    var id: String {
        swap self {
        case .error(let message):
            return message
        case .success(let message):
            return message
        case .affirmation:
            return "affirmation"
        }
    }
    
    var alert: Alert {
        swap self {
        case .error(let message):
            return Alert(title: Textual content("Error"), message: Textual content(message), dismissButton: .default(Textual content("OK")))
        case .success(let message):
            return Alert(title: Textual content("Success"), message: Textual content(message), dismissButton: .default(Textual content("OK")))
        case .affirmation(let motion):
            return Alert(
                title: Textual content("Affirmation"),
                message: Textual content("Are you positive?"),
                primaryButton: .damaging(Textual content("Delete"), motion: motion),
                secondaryButton: .cancel()
            )
        }
    }
}

NewPostView.swift


import SwiftUI
import Firebase
import FirebaseFirestore
import FirebaseStorage
import CoreLocation
import Imaginative and prescient
import CoreML

struct NewPostView: View {
    u/EnvironmentObject var userSession: UserSession
    u/EnvironmentObject var viewRouter: ViewRouter
    u/State personal var showImagePicker: Bool = false
    u/State personal var usingCamera: Bool = false
    u/State personal var postImage: UIImage?
    u/State personal var postDescription: String = ""
    u/State personal var postHashtags: String = ""
    u/State personal var postLocation: String = ""
    u/State personal var sneakerModel: String = ""
    u/State personal var additionalDetails: String = ""
    u/State personal var locationManager = CLLocationManager()
    u/State personal var activeAlert: NewPostAlertState?
    u/State personal var isUploading: Bool = false
    u/State personal var showConfirmationAlert: Bool = false
    u/State personal var showInvalidImageAlert: Bool = false 

    u/State personal var modelLoaded: Bool = false

    var physique: some View {
        NavigationView {
            VStack {
                if !modelLoaded {
                    Textual content("Did not load the mannequin. Please verify your MLModel file.")
                        .foregroundColor(.crimson)
                        .padding()
                } else {
                    ScrollView {
                        VStack(spacing: 10) {
                            Textual content("Add Picture")
                                .font(.headline)
                                .padding(.prime)
                            
                            imageSection
                                .body(top: 250)
                                .padding(.horizontal)
                            
                            Textual content("Submit Particulars")
                                .font(.headline)
                                .padding(.prime)

                            postDetailsSection
                                .padding(.horizontal)
                        }
                        .padding(.horizontal, 16)
                    }

                    Spacer()

                    postButton
                        .padding()
                        .background(
                            LinearGradient(gradient: Gradient(colours: [Color.white, Color.brown.opacity(0.3)]), startPoint: .prime, endPoint: .backside)
                                .edgesIgnoringSafeArea(.backside)
                        )
                }
            }
            .navigationTitle("New Submit")
            .navigationBarItems(main: Button(motion: {
                viewRouter.goToMain()
            }) {
                HStack {
                    Picture(systemName: "chevron.backward")
                    Textual content("Again")
                }
            })
            .background(
                LinearGradient(gradient: Gradient(colours: [Color.white, Color.brown.opacity(0.3)]), startPoint: .prime, endPoint: .backside)
                    .edgesIgnoringSafeArea(.all)
            )
            .onAppear {
                loadModel()
            }
            .sheet(isPresented: $showImagePicker) {
                ImagePicker(picture: $postImage, sourceType: usingCamera ? .digicam : .photoLibrary)
            }
            .alert(merchandise: $activeAlert) { alertState in
                alertState.alert
            }
            .alert(isPresented: $showConfirmationAlert) {
                Alert(
                    title: Textual content("Is that this a sneaker picture?"),
                    message: Textual content(""),
                    primaryButton: .default(Textual content("Sure"), motion: validateAndUploadImage),
                    secondaryButton: .cancel(Textual content("No"), motion: {
                        print("Picture rejected by person")
                        activeAlert = .error("The chosen picture just isn't acknowledged as a sneaker.")
                    })
                )
            }
            .alert(isPresented: $showInvalidImageAlert) { // Alerta para imagen inválida
                Alert(
                    title: Textual content("Invalid Picture"),
                    message: Textual content("The chosen picture just isn't acknowledged as a sneaker. Please choose one other picture."),
                    dismissButton: .default(Textual content("OK"))
                )
            }
        }
    }

    personal func loadModel() {
        if let _ = strive? VNCoreMLModel(for: SneakerClassifier().mannequin) {
            print("Mannequin loaded efficiently")
            modelLoaded = true
        } else {
            print("Did not load the mannequin")
            modelLoaded = false
        }
    }

    personal var imageSection: some View {
        Button(motion: {
            showImagePicker = true
            usingCamera = false
        }) {
            ZStack {
                RoundedRectangle(cornerRadius: 15)
                    .fill(Coloration.grey.opacity(0.2))
                    .body(maxWidth: .infinity)
                    .overlay(RoundedRectangle(cornerRadius: 15).stroke(Coloration.grey, lineWidth: 2))

                if let picture = postImage {
                    Picture(uiImage: picture)
                        .resizable()
                        .scaledToFit()
                        .clipShape(RoundedRectangle(cornerRadius: 15))
                } else {
                    VStack {
                        Picture(systemName: "digicam.fill")
                            .resizable()
                            .scaledToFit()
                            .body(width: 80, top: 80)
                            .foregroundColor(.grey)
                            .shadow(radius: 10)
                        Textual content("Faucet so as to add picture")
                            .foregroundColor(.grey)
                    }
                }
            }
        }
        .padding()
    }

    personal var postDetailsSection: some View {
        VStack(spacing: 10) {
            inputField("Write Description", textual content: $postDescription, icon: "pencil")
            inputField("Sneaker Mannequin", textual content: $sneakerModel, icon: "tag")
            inputField("Extra Particulars", textual content: $additionalDetails, icon: "information.circle")
            
            VStack(alignment: .main, spacing: 8) {
                HStack {
                    Picture(systemName: "quantity")
                        .foregroundColor(.grey)
                    TextField("Add hashtags", textual content: $postHashtags, onCommit: addHashtag)
                        .textFieldStyle(RoundedBorderTextFieldStyle())
                        .padding(.main, 5)
                        .body(top: 30)
                }
                .padding()
                .background(Coloration.white)
                .cornerRadius(10)
                .shadow(radius: 5)
                
                Textual content("Hashtags: (formattedHashtags)")
                    .font(.caption)
                    .foregroundColor(.grey)
                    .padding(.main)
            }

            locationField
                .padding()
        }
    }

    personal func inputField(_ placeholder: String, textual content: Binding<String>, icon: String) -> some View {
        HStack {
            Picture(systemName: icon)
                .foregroundColor(.grey)
            TextField(placeholder, textual content: textual content)
                .textFieldStyle(RoundedBorderTextFieldStyle())
                .body(top: 30)
        }
        .padding()
        .background(Coloration.white)
        .cornerRadius(10)
        .shadow(radius: 5)
    }

    personal var locationField: some View {
        Button(motion: fetchLocation) {
            HStack {
                Picture(systemName: "location.fill")
                Textual content(postLocation.isEmpty ? "Add Location" : postLocation)
            }
            .padding()
            .foregroundColor(.main)
            .background(Coloration.blue.opacity(0.2))
            .cornerRadius(10)
            .shadow(radius: 5)
            .animation(.easeInOut)


        }
    }

    personal var postButton: some View {
        Button(motion: {
            print("Submit button tapped")
            guard validateFields() else {
                print("Validation failed")
                activeAlert = .error("Please fill in all fields.")
                return
            }
            print("Fields validated")
            showConfirmationAlert = true
        }) {
            if isUploading {
                ProgressView()
                    .progressViewStyle(CircularProgressViewStyle(tint: .white))
                    .body(width: 20, top: 20)
            } else {
                Textual content("Submit")
                    .body(maxWidth: .infinity)
            }
        }
        .buttonStyle(GradientButtonStyle())
        .padding()
    }

    personal func fetchLocation() {
        locationManager.requestWhenInUseAuthorization()
        if let location = locationManager.location {
            let geocoder = CLGeocoder()
            geocoder.reverseGeocodeLocation(location) { locations, _ in
                postLocation = locations?.first?.locality ?? "Unknown location"
            }
        }
    }

    personal func validateAndUploadImage() {
        guard let picture = postImage else {
            print("No picture to validate")
            activeAlert = .error("Please choose a picture.")
            return
        }

        print("Validating picture")
        // Validar la imagen con el modelo
        validateSneakerImage(picture: picture) { isValid in
            if isValid {
                print("Picture is legitimate")
                uploadImageAndPost()
            } else {
                print("Picture is invalid")
                showInvalidImageAlert = true // Mostrar alerta si la imagen no es válida
            }
        }
    }

    personal func validateSneakerImage(picture: UIImage, completion: u/escaping (Bool) -> Void) {
        guard let mannequin = strive? VNCoreMLModel(for: SneakerClassifier().mannequin) else {
            print("Did not load mannequin for validation")
            completion(false)
            return
        }

        let request = VNCoreMLRequest(mannequin: mannequin) { request, error in
            guard let outcomes = request.outcomes as? [VNClassificationObservation],
                  let firstResult = outcomes.first else {
                print("No legitimate outcomes from mannequin")
                completion(false)
                return
            }
            print("Validation end result: (firstResult.identifier) with confidence (firstResult.confidence)")
            completion(firstResult.identifier == "sneaker" && firstResult.confidence > 0.8)
        }

        guard let ciImage = CIImage(picture: picture) else {
            print("Did not convert UIImage to CIImage")
            completion(false)
            return
        }

        let handler = VNImageRequestHandler(ciImage: ciImage, choices: [:])
        DispatchQueue.world(qos: .userInitiated).async {
            do {
                strive handler.carry out([request])
                print("Picture classification request carried out")
            } catch {
                print("Did not carry out classification: (error.localizedDescription)")
                completion(false)
            }
        }
    }

    personal func uploadImageAndPost() {
        guard let picture = postImage, let imageData = picture.jpegData(compressionQuality: 0.8) else {
            print("Failed to arrange picture for add")
            activeAlert = .error("Failed to arrange picture.")
            return
        }

        print("Importing picture...")
        isUploading = true

        let storageRef = Storage.storage().reference().little one("sneaker_images/(UUID().uuidString).jpg")
        storageRef.putData(imageData, metadata: nil) { metadata, error in
            if let error = error {
                print("Error importing picture: (error.localizedDescription)")
                activeAlert = .error("Error importing picture.")
                isUploading = false
                return
            }

            print("Picture uploaded, fetching obtain URL...")
            storageRef.downloadURL { url, error in
                if let error = error {
                    print("Error retrieving picture URL: (error.localizedDescription)")
                    activeAlert = .error("Error retrieving picture URL.")
                    isUploading = false
                    return
                }

                guard let downloadURL = url else {
                    print("No obtain URL discovered")
                    activeAlert = .error("No obtain URL discovered.")
                    isUploading = false
                    return
                }
                print("Picture obtain URL retrieved: (downloadURL.absoluteString)")
                createPost(imageUrl: downloadURL.absoluteString)
            }
        }
    }

    personal func createPost(imageUrl: String) {
        guard let person = userSession.person else {
            print("Person session not discovered")
            activeAlert = .error("Person session not discovered.")
            isUploading = false
            return
        }

        let hashtagsArray = postHashtags.break up(separator: " ").map { String($0).trimmingCharacters(in: .whitespacesAndNewlines) }
        let newPost = SneakerPost(
            imageUrl: imageUrl,
            uploader: person.nickname,
            nickname: person.nickname,
            sneakerModel: sneakerModel,
            description: postDescription,
            hashtags: hashtagsArray,
            uploadDate: Date(),
            replies: [],
            additionalDetails: additionalDetails,
            likes: [],
            feedback: []
        )

        print("Creating submit...")
        userSession.addNewPost(newPost) { success in
            isUploading = false
            if success {
                print("Submit created efficiently")
                viewRouter.goToMain()
            } else {
                print("Failed to avoid wasting submit")
                activeAlert = .error("Failed to avoid wasting submit.")
            }
        }
    }

    personal func addHashtag() {
        let trimmedHashtag = postHashtags.trimmingCharacters(in: .whitespacesAndNewlines)
        guard !trimmedHashtag.isEmpty else { return }

        var hashtags = postHashtags.break up(separator: " ").map(String.init)

        if !trimmedHashtag.hasPrefix("#") {
            hashtags.append("#" + trimmedHashtag)
        } else {
            hashtags.append(trimmedHashtag)
        }

        postHashtags = hashtags.filter { !$0.isEmpty }.joined(separator: " ")
    }

    personal func validateFields() -> Bool {
        return !postDescription.isEmpty &&
            !postHashtags.isEmpty &&
            !postLocation.isEmpty &&
            !sneakerModel.isEmpty &&
            !additionalDetails.isEmpty &&
            postImage != nil
    }

    personal var formattedHashtags: String {
        postHashtags.break up(separator: " ").map { $0.hasPrefix("#") ? $0 : "#($0)" }.joined(separator: " ")
    }
}

struct NewPostView_Previews: PreviewProvider {
    static var previews: some View {
        NewPostView().environmentObject(UserSession()).environmentObject(ViewRouter())
    }
}

Mannequin Loading: The mannequin appears to load appropriately based mostly on the console output “Mannequin loaded efficiently”. Is there any extra configuration or verification required for the mannequin?

Submit Creation Stream: Regardless of reaching the validation step (“Fields validated”), the submit creation course of doesn’t proceed. What could possibly be inflicting the blockage, and the way can I debug this additional?

CoreGraphics Warnings: There are a number of warnings about passing an invalid numeric worth to CoreGraphics. May this be associated to my concern, and the way can I resolve these warnings?

Firebase Setup: Are there any particular Firebase configurations or permissions that is perhaps stopping the submit creation from succeeding?

Any assist or insights can be drastically appreciated. Thanks!



Supply hyperlink

RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

- Advertisment -
Google search engine

Most Popular

Recent Comments